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I.  INTRODUCTION 


A.  [ PURPOSE 

I 

\y  This  report  presents  an  evaluation  of  the  Tri-Service  Laboratory 
(TRILAB)  System  -.Department  of  the  Navy  at  the  Naval 

Regional  Medical  Center  (NRMC)  Oakland.  This  evaluation  is^based  on 
comparison  of  post-implementation  (Y-Period)  data  with  baseline  (X 

Period)  data  collected  at  the  facility.  _ _ _ 

TRILAB  is  being  implemented  at  three  sites:  /*’ 

•  Naval  Regional  Medical  Center  (NRMC)  Oakland,  California; 

•  Dwight  D.  Eisenhower  Army  Medical  Center  (DDEAMC) , 

Augusta,  Georgia;  and 

•  Wright  Patterson  Medical  Center  and  Regional  Hospital, 

Dayton,  Ohio. 

NRMC  Oakland  was  the  first  facility  to  receive  the  TRILAB  system 

and  was  chosen  as  the  primary  evaluation  site  for  the  system.  The 

original  evaluation  plan  for  the  TRILAB  system  was  developed  by 

Analytic  Services,  Inc.  (ANSER) , 1  in  1980.  Baseline  data  were 

collected  under  the  supervision  of  ANSER  staff  during  an  eight-week 

period  (September  29,  1980-November  29,  1980).  Some  of  the  data  were 

2 

analyzed  by  ANSER  and  summarized  in  their  progress  report.  The 

baseline  data  and  progress  report  were  provided  to  Arthur  D.  Little, 

Inc.  (ADL) .  An  assessment  of  the  baseline  data  has  been  reported 

3 

previously  in  a  six-volume  baseline  report. 

This  evaluation  report  concerning  TRILAB  at  NRMC  is  the  second 
volume  in  a  four-volume  report.  This  report  is  presented  in  four 
chapters: 

•  the  remaining  sections  of  this  chapter  describe  the 
TRILAB  system  and  operations  of  the  clinical  laboratory 
at  NRMC  Oakland  during  the  baseline  period  and  under 
the  TRILAB  system; 

•  the  evaluation  approach  and  data  collection  methodology 
are  described  in  Chapter  II; 
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•  Chapter  III  presents  the  results  of  the  post-implementation 
analysis  and  a  comparison  of  results  with  the  baseline 
data;  and 

•  the  final  chapter  compares  the  results  obtained  versus 
the  original  goals  set  for  the  system  by  the  Medical 
Review  Group  (MRG) . 

B.  DESCRIPTION  OF  TRILAB 

The  TRILAB  system  is  designed  to  support  the  following  laboratory 
activities:  patient  files,  test  order  entry,  specimen  accessioning 

and  control,  work  document  preparation,  quality  control,  test  result 
entry,  inquiry  and  data  retrieval,  test  result  reporting  at  wards  and 
clinics,  and  management  reporting. 

The  TRILAB  system  is  designed  to  have  automated,  high-volume  test 
instruments  on-line,  with  the  goal  of  significantly  reducing  clerical 
work  of  laboratory  technicians  and  transcription  errors.  The  system 
is  also  designed  to  monitor  quality  control  samples  in  order  to  check 
for  correct  calibration  of  instruments  and  proper  handling  of  the 
specimens  within  the  laboratory,  and  to  produce  interim  test  results 
reports,  daily  cumulative  reports,  and  cumulative  summary  discharge 
reports. 

In  addition,  the  system  produces  management  information,  such  as 
laboratory  workload  summary  reports,  which  should  reduce  the  effort  to 
prepare  management  reports  and  assist  in  the  efficient  organization 
and  administration  of  the  laboratory,  . . — 

The  TRILAB  system,  which  uses  software  developed  by  Meditech  and 
was  obtained  from  Centennial  Systems  Corporation,  through  competitive 
procurement.  The  system  can  support  terminals  outside  the  laboratory, 
such  as  in  wards,  clinics  and  satellite  facilities  for  transmission  of 
results  and  for  inquiry  as  to  test  status. 

In  addition  to  making  results  available  via  terminal  inquiry,  the 
TRILAB  system  produces  a  number  of  "hard-copy"  (printed)  reports: 

(1)  Summary  Report  by  Location  (SL).  This  is  a  cumulative 

summary  report,  organized  by  inpatient  location,  printed 
overnight  and  picked  up  at  about  0400,  to  be  available 
for  physicians'  morning  rounds. 
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(2)  Report  by  Doctor  (RD) .  This  is  the  largest  report 
produced.  It  is  a  listing  of  daily  outpatient  results 
for  inclusion  in  the  outpatient  medical  record.  This 
is  sorted  by  physician’s  name  and  location,  and  then 
by  terminal  patient  digit  to  facilitate  filing  by 
Outpatient  Medical  Records,  which  receives  this  report. 

(3)  Summary  of  Discharges  (SD).  This  report,  which  is 
produced  at  inpatient  discharge,  summarizes  ail  the 
patient's  results  for  inclusion  in  the  inpatient  chart. 

It  is  ordered  by  inpatient  terminal  digit  and  is  designed 
to  replace  the  individual  daily  laboratory  reports.  It 
is  organized  one  page  per  laboratory  section,  with  all 
results  for  each  type  of  test  ordered  by  date  from  left 
to  right  in  a  single  line. 

(4)  Outpatient  Requesting  Report  (OPR) .  This  report  is  sorted 
by  outpatient  requesting  location,  physician,  and  alpha¬ 
betically  by  patient.  In  consists  of  one  page  per  patient, 
is  not  designed  to  be  filed  in  the  chart,  and  is  sent  to  the 
requesting  clinics. 

(5)  Outstanding  Specimen  Report  (OSR) .  This  is  an  internal 
laboratory  management  report,  sorted  by  section. 

(6)  Review  Results  Log  (Micro) .  This  is  essentially  a  patient 
listing  log  for  the  Cytology  section,  which  must  maintain 
a  hard-copy  log  for  epidemiological  purposes. 

(7)  Report  by  Location  (RL).  This  is  an  interim  daily  report 
for  inpatients,  similar  in  format  to  the  OPR,  but  not 
cumulative,  and  is  designed  only  for  those  inpatient  units 
which  may  have  pre-operative  patients.  It  is  available  at 
1900. 

(8)  Levey-Jennings .  This  report  is  printed  on  weekends 
when  system  usage  is  low  and  other  reports  are  not 
produced.  This  report  consists  of  plots  and  results 
obtained  for  control  tests  by  day  for  the  past  month, 
and  the  standard  deviations  identified  so  that  number 

and  trends  of  lutliers  can  be  easily  (visually)  identified. 


C.  DESCRIPTION  OF  OPERATIONS  OF  THE  CLINICAL  LABORATORY  AT  NRMC 
OAKLAND 
1.  Overview 

NRMC  Oakland  is  a  large  medical  treatment  facility  (MTF) ,  with  an 
average  daily  inpatient  census  of  244  patients  and  service  volume  of 
376,326  outpatient  visits  per  year  (FY  1982).  This  compares  with  an 
average  census  of  234  and  outpatient  volume  of  396,621  visits  in  FY 
1980.  During  FY  1982,  the  clinical  laboratory  performed  approxi¬ 
mately  2,586,000  tests  (including  quality  controls)  compared  with 
2,642,000  tests  in  1980.  Laboratory  workload  has  thus  remained 
relatively  stable  between  the  baseline  and  post-implementation 
periods.  During  October  1982,  when  the  post-implementation  data 
collection  effort  was  carried  out,  the  laboratory  performed  170,150 
patient  tests  (excluding  quality  control).  This  is  15  percent  greater 
than  the  147,625  tests  reported  performed  in  October  1980  (when  the 
baseline  survey  was  carried  out) . 

Clinical  laboratory  staffing  during  the  two  periods  was  as 


follows : 

April 

Authorized 

1980 

Assigned 

October 

Authorized 

1982 

Assigned 

Pathologists 

6 

8 

6 

5 

Pathology  residents 

8 

9 

8 

5 

Medical  Service  Corps 
(MSC)  laboratory  officers 

5 

5 

5 

5 

Military  laboratory 
technicians 

59 

41 

52 

44 

Civilian  laboratory 
technicians 

23 

23 

^  — T 

i 

29 

Civilian  receptionists/ 
secretaries 

_ 4 

_4 

_ 4 

*- 

Total 

105 

90 

102 

90 

The  assigned  staff  of  90  includes  71  technical  personnel.  In 
1980  there  were  also  90  assigned  personnel,  including  67  technical 
personnel . 
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The  laboratory  operates  during  all  shifts,  all  days  of  the  week. 
However,  the  staffing  is  greatly  reduced  on  nights,  weekends,  ana 
holidays.  The  work  day  is  divided  into  three  shifts,  with  the  primary 
responsibility  for  the  evening  and  night  shifts  being  the  handling  of 
urgent  and  emergency  tests  requests. 

The  laboratory  is  divided  into  six  major  sections: 

•  Hematology/Urinalysis  (includes  Special  Hematology) , 

•  Chemistry, 

•  Special  Chemistry  (located  outside  the  main  facility), 

•  Blood  Bank, 

•  Microbiology  (includes  Mycology,  Serology,  and 
Parasitology) ,  and 

•  Anatomical  Pathology  (includes  Histopathology , 

Autopsy,  and  Cytology). 

Additionally,  laboratory  test  procedures  are  performed  in  the 
Radio- Immunoassay  Laboratory  (staffed  by  Nuclear  Medicine  Department 
personnel  and  located  in  the  clinical  laboratory)  and  in  the 
Cardiopulmonary  Laboratory  (staffed  by  Medicine  Department  personnel 
and  located  on  the  fourth  floor  of  the  facility). 

2,  Baseline  Work  Flow 

The  typical  procedure  for  handling  test  requests  and  reports 
prior  to  implementation  of  TRILAB  is  described  below.  Figure  1 
presents  a  flow  diagram  schematic  of  the  information  and  test  flow. 

An  authorized  health  care  provider  (HCP)  ordered  laboratory  test 
procedures  using  order  requisition  forms — generally  one  for  each 
section  of  the  laboratory.  Authorized  HCPs  (those  who  may  order 
laboratory  procedures  for  a  patient)  included  physicians,  dentists, 
nurse  practitioners,  and  physician  assistants.  The  ordering  HC?  could 
prepare  the  test  request  slip,  or  the  HCP  could  delegate  this  task  to 
a  nurse,  corpsman,  or  unit  clerk. 
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Outpatients  and  ambulatory  inpatients  carried  the  prepared  test 
request  slip  to  the  laboratory  where  the  required  specimens(s)  were 
collected.  The  laboratory  receptionists  controlled  specimen 
collection  for  outpatients  and  ambulatory  inpatients.  Specimen 
collection  for  nonambulatory  inpatients  was  performed  on  the  ward, 
mainly  by  ward  personnel,  with  subsequent  transport  of  the  specimen 
and  request  slip  to  the  laboratory  by  ward  personnel.  (This  part  of 
the  system  is  the  same  with  TRILAB  as  under  manual  operations.) 

After  the  test  request  slip  and  specimen  were  brought  to  the 
appropriate  laboratory  section,  a  technician  accessioned  and  logged 
each  specimen  received  or  collected  by  the  section.  Depending  upon 
the  test  priority  stated  by  the  HCP  and  indicated  on  the  slip,  a 
technician  processed  the  test  immediately  or  at  a  later  time.  The 
three  types  of  test  priority  were  routine  (normal  processing),  urgent 
or  "ASAP"  (results  required  faster  than  normal),  and  emergency  (STAT) 
results  required  immediately).  For  each  instrument  run,  a  "worksheet" 
was  prepared,  which  listed  the  sequence  of  patient  specimens  to  be 
processed  and  the  tests  to  be  performed  on  each.  The  laboratory 
technician,  having  performed  the  appropriate  test  procedures,  arrived 
at  a  preliminary,  uncertified  result.  The  results  were  copied  onto 
the  worksheet.  Following  certification  (validation)  of  the  preliminary 
result  by  the  laboratory  section  supervisor,  the  certified  test  result 
was  communicated  to  the  requesting  location.  Laboratory  personnel 
immediately  telephoned  certified  results  of  urgent  and  emergency  tests 
to  the  requesting  HCP. 

Results  of  all  tests  were  recorded  in  the  multiple-copy  requi¬ 
sition  form,  which  was  separated  and  routed  to  various  destinations 
within  the  facility.  The  routing  sequence  differed  for  outpatients 
and  inpatients.  For  outpatients,  the  top  copy  (the  "original")  of  the 
slip  went  to  Outpatient  Medical  Records  for  filing  in  the  patient's 
Outpatient  Health  Record;  the  second  copy  of  the  slip  went  to  the 
clinic  requesting  the  test  procedure  (where  the  HCP  reviewed  it);  and 
the  third  copy  of  the  slip  was  retained  and  filed  in  laboratory  files 
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located  in  the  reception  area.  For  inpatients,  the  top  copy  of  the 
slip  was  sent  to  the  requesting  ward,  where  the  HCP  reviewed  it  and 
the  slip  was  incorporated  into  the  patient's  Inpatient  Health  Record; 
the  second  copy  of  the  slip  was  discarded;  and  the  third  copy  of  the 
slip  was  retained  and  filed  in  the  laboratory. 

3.  TRILAB  Configuration 

The  TRILAB  system  obtained  for  NRMC  Oakland  included  a  mainframe 
computer  with  512  KB  memory,  two  96  MB  disks  and  an  800  BP1  tape  unit. 
Peripherals  included  42  CRTs,  seven  character  printers,  and  one  line 
printer.  Eighteen  of  the  terminals  were  located  in  patient  care 
areas.  Six  terminals  were  located  in  outpatient  areas:  the  Emergency 
Room,  and  the  Obstetric,  Cardiopulmonary,  Medical/Dermatology/- 
Cardiology,  Surgical  and  Primary  Care  Clinics.  Twelve  terminals  were 
located  in  inpatient  areas:  The  CCU,  8W,  Recovery,  ICU,  PICU,  6N,  6W, 
9,  7N,  9W,  7E  and  8S  units.  Two  CRTS  were  located  in  Inpatient 
Admissions,  two  in  the  Outpatient  Admissions  area  and  the  remainder  in 
the  laboratory.  The  line  printer  and  one  character  printer  were 
located  in  the  computer  room;  the  remaining  six  printers  were  located 
in  the  laboratory  area. 

The  only  instruments  which  were  on-line  to  the  computer  were  the 
SMAC,  Coulter  S+,  and  Clintech,  although  interfaces  were  also  obtained 
for  the  Gemini,  Coagamate,  and  electrolyte  instruments. 

4.  Post-Implementation  Work  Flow 

Figure  2  presents  a  flow  diagram  schematic  of  the  information  and 
test  flow  during  the  post-implementation  period.  Preparation  of 
laboratory  test  requests  and  transportation  of  specimens  to  the 
laboratory  was  the  same  as  in  the  baseline  period. 

Within  each  laboratory  section,  a  technician  "accessioned"  each 
test  request  via  terminal  in  each  section  by  looking  up  the  patient's 
record  and  entering  the  requested  tests.  In  most  instances  the 
patient's  record  should  be  in  the  file,  having  been  pre-admitted  by 
Outpatient  or  Inpatient  Admissions.  In  those  cases  where  a  patient’s 
name  was  not  in  the  file,  the  laboratory  technician  created  a  patient 
record,  including  required  demographic  information.  The  computer  was 
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then  utilized  to  generate  labels  where  appropriate  (e.g.,  in 
Microbiology)  and  worksheets,  which  list  for  each  instrument  the 
patients  and  required  tests. 

Tests  were  performed  in  the  following  order  of  priority: 

•  "emergency"  or  STAT  (results  required  immediately); 

•  urgent  or  "ASAP"  results  (results  required  faster  than 
normal) ; 

•  routine  inpatient;  and 

•  routine  outpatient. 

For  those  instruments  and  tests  which  were  not  on-line  to  the 
computer,  the  results  were  entered  manually  on  the  worksheet  and  then 
entered  into  the  computer  via  a  terminal,  in  worksheet  format.  For 
the  on-line  instruments,  the  results  were  automatically  entered  in  the 
computer. 

The  results  were  then  reviewed  via  a  "Results  Review  Program" — 
that  is,  verified  by  supervisory  personnel — and  were  then  available 
for  requester  inquiry  via  terminal  at  any  location  where  the  terminals 
were  located.  In  the  case  of  STAT/urgent  requests,  the  terminal  in 
the  requester's  location  "beeped"  to  inform  the  unit  or  clinic  that  a 
STAT  result  was  available.  When  a  request  slip  included  multiple 
tests  some  of  which  were  "emergency"  or  "STAT,"  all  tests  on  the  slip 
received  this  designation,  but  only  the  true  emergency  or  STAT  tests 
were  "beeped"  back  on  completion,  with  the  remaining  tests  receiving 
routine  handling. 

The  "interim  results  report"  was  printed  by  the  computer  in  the 
late  afternoon  for  some  LO  inpatient  units  which  were  likely  to  have 
pre-operative  patients.  This  report,  scheduled  to  be  available  by 
1900  hours,  consisted  of  one  page  per  patient  and  included  the  results 
of  tests  performed  during  the  previous  24  hours.  During  the  night 
shift  the  daily  summary  inpatient  and  outpatient  reports  were  printed, 
which  were  picked  up  at  around  0400  hours  for  dissemination  to 
inpatient  units  and  outpatient  clinics. 

As  mentioned  earlier,  a  number  of  other  reports  were  produced  by 
the  TRILAB  system,  including  the  "summary  of  discharge"  report,  which 
summarized  all  patient's  results  for  inclusion  in  the  inpatient  chart 
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on  discharge;  the  "outstanding  specimen"  report,  which  was  utilized 
internally  by  laboratory  management  to  follow  up  on  specimens  for 
which  completed  results  were  not  yet  available;  the  "review  results 
log"  for  Microbiology;  the  "Levey-Jennings  report,"  which  was  printed 
in  weekends  and  was  utilized  by  the  laboratory  for  quality  control 
analysis;  and  the  "workload  summary"  report,  which  summarized  tests 
performed  by  each  section. 

The  major  differences  between  the  baseline  (manual)  system  and 
the  post-implementation  (TRILAB)  system  were: 

•  Providers  in  locations  provided  with  terminals 
received  results  via  terminal  as  soon  as  they 
were  available,  instead  of  having  to  wait  for 
telephone  calls  (in  the  case  of  STAT  and  urgent 
requests)  or  the  completed  test  request  slip  (in  the 
case  of  routine  requests). 

•  the  reports  received  by  providers  were  cumulative, 
formatted  and  on  full-size  paper,  instead  of  the 
original  request  slips.  Medical  Records  received 

a  single  cumulative  discharge  report  for  inpatients, 
rather  than  a  number  of  slips  for  each  day's  tests 
that  were  carried  out  during  a  patient's  stay. 

•  Within  the  laboratory,  work  sheets  were  prepared  by  the 
computer  after  tests  had  been  accessioned,  instead  of 
being  prepared  manually. 

•  For  those  instruments  which  were  not  on-line  to  the 
computer,  technicians  entered  results  into  the  computer 
memory  via  terminal,  instead  of  writing  the  results  on 
the  original  test  request  slips.  For  those  instruments 
which  were  on-line,  the  results  were  automatically 
entered  into  the  computer  memory.  Review  of  test  results 
was  expedited  in  that  normal  values  and  outliers  were 
automatically  identified,  as  well  as  unusual  changes 
from  previous  patients'  test  results. 

•  The  computer  produced  a  number  of  management  reports  for 
the  laboratory  which  previously  were  prepared  manually. 
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•  Test  status  and  results  could  be  obtained  via  inquiry 
from  any  terminal  connected  to  the  system,  instead  of  by 
looking  up  records  or  phoning  the  laboratory. 
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II.  METHODOLOGY 

This  chapter  presents  a  summary  of  the  baseline  and 

post-implementation  period  evaluation  plans  for  NRMC  Oakland.  A  more 

complete  description  of  the  baseline  evaluation  plan  is  presented  in 

the  report,  "Baseline  Evaluation  of  the  Tri-Service  Laboratory  System 
3 

Volume  II,"  and  of  the  post-implementation  evaluation  plan,  in  the 
report  "Plan  for  the  Y  Period  Evaluation  of  the  Tri-Service  Laboratory 
System. 

The  baseline  evaluation  plan  for  the  TRILAB  system,  developed  by 

1  ♦ 

Analytic  Services,  Inc.  (ANSER)  described  36  hypotheses  regarding  the 
potential  impacts  of  TRILAB  on  the  clinical  laboratory  and  the  MTF. 
These  were  grouped  into  the  following  four  areas: 

•  personnel  time; 

•  satisfaction  and  perception; 

•  information  attributes;  and 

•  costs. 

The  evaluation  called  for  a  before-and-af ter  study  comparing 
manual  laboratory  operation  with  operation  of  the  laboratory  using  the 
TRILAB  computer  system.  Two  periods  of  data  collection  were  planned: 
the  baseline  period  with  manual  operation  before  the  installation  of 
TRILAB;  and  the  post-implementation  when  TRILAB  was  operational.  This 
before-and-af ter  approach  was  followed  in  the  plans  for  the  three 
TRILAB  evaluations. 

Table  1  summarizes  the  evaluation  elements  and  data  collection 
methods  used  in  the  baseline  evaluation  at  NRMC  Oakland.  Four  types 
of  data  were  collected  by  ANSER  during  the  baseline  study  at  NRMC 
Oakland: 

•  time  spent  by  personnel  within  the  laboratory  in 
information  handling  activities  (using  work  sampling 
and  timed  observations); 

•  performance  of  services  (turnaround  time  for  test  results 
in  the  laboratory;  transcription  discrepancies;  number  of 
telephone  inquiries  about  test  results;  and  patient  waiting 
time) ; 
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•  staff  perceptions  of  performance  of  services  (staff 
questionnaire  survey);  and 

•  staff  and  patient  satisfaction  (staff  and  patient 
questionnaire  survey). 

The  post-implementation  evaluation  plan  was  developed  with  the 
following  considerations: 

(1)  In  order  to  utilize  the  baseline  data  to  the  maximum 
extent,  and  to  make  the  before-and-af ter  comparison 
as  consistent  and  meaningful  as  possible,  the  same 
evaluative  measures,  data  collection  methodologies, 
and  data  collection  instruments  used  in  the  baseline 
period  were  used  in  the  post-implementation  study  to 
the  extent  possible.  It  was  necessary,  however,  to 
modify  the  data  collection  methodology  and  instruments 
in  several  instances,  in  view  of  the  differences  in 
procedures  in  the  laboratory  because  of  the  TRILAB 
system. 

(2)  Sample  sizes  were  chosen  to  provide  as  reliable  as 

possible  data,  given  the  data  available  from  the 

2 

baseline  evaluation. 

(3)  Where  the  baseline  data  were  unavailable  or  insuf¬ 
ficient  for  evaluation  purposes,  an  effort  was  made 
to  collect  such  information  via  interviews  and  review 
of  any  available  reports. 

The  post-implementation  data  collection  plan  was  organized  under 
the  following  data  collection  methodologies: 

(1)  Work  sampling  at  NRMC  Oakland,  to  collect  data  on 
distribution  of  moratory  personnel  activities  in 
the  post-implementation  period  for  comparison  with 
distribution  of  activities  in  the  baseline.  The 
major  objective  was  to  determine  whether  time  spent 
in  information  handling  activities  had  changed. 

(2)  Process  time  study  at  NRMC  Oakland  and  DDEAMC  to 
determine  whether  process  times  for  return  of  test 
results  had  changed. 
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(3)  Volume  of  celephone  calls  to  the  laboratory  at  NRMC 
Oakland  and  DDEAMC,  to  determine  whether  changes  had 
occurred  in  the  volume  of  telephone  inquiries  to  the 
laboratory  as  a  result  of  the  availability  of  results 
via  the  TRILA3  system. 

(4)  Surveys  of  laboratory  staff,  providers,  and  patients, 
to  determine  whether  satisfaction  with  clinical  labora¬ 
tory  services  had  changed. 

(5)  Interviews  and  supplementary  data  collection,  to  obtain 
cost  data,  volume  data,  and  other  information. 

Table  2  summarizes  the  relationship  between  the  evaluation 
elements  and  the  data  obtained  in  the  post-implementation  study. 

The  basic  data  for  the  post-implementation  evaluation  at  NRMC 
Oakland  were  collected  during  an  intensive  study  period,  October  8-22, 
1982.  (No  data  were  collected  on  Columbus  Day,  October  11,  since  the 
holiday  was  considered  unrepresentative  of  normal  laboratory 
activities.)  The  data  collection  period  was  thus  two  years  after  the 
baseline  data  collection  (October-November  1980)  and  eight  months 
after  initiation  of  the  TRILAB  system  implementation.  In  addition  to 
data  collection  and  interviews  during  this  study  period,  three 
implementation  monitoring  visits  were  made  prior  to  the  study  period 
(in  March,  May  and  August  1982),  during  which  laboratory  staff  and 
providers  were  interviewed.  The  results  of  these  interviews  were  also 
used  in  the  evaluation. 
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III.  RESULTS 


A.  INTRODUCTION 

This  chapter  summarizes  and  compares  the  results  of  the  baseline 
and  post-implementation  studies  at  NRMC  Oakland.  The  evaluation  is 
organized  under  the  following  topics: 

•  Personnel  Time  Devoted  to  Information  Handling; 

•  Service  Performance  (turnaround  times,  number 
of  telephone  calls) ; 

•  Staff  Perceptions  of  TRILAB; 

•  Patient  Satisfaction  with  Laboratory  Services; 

•  System  Costs. 

The  comparison  of  the  results  obtained  in  the  baseline  and 
post-implementation  periods  is  complicated  by  the  following  factors: 

•  The  baseline  and  post-implementation  data  collection 
studies  were  conducted  by  different  contractors;  the 
original  evaluation  plan,  baseline  data  collection  and 
some  of  the  baseline  data  analysis  were  performed  by 
ANSER,  while  the  remainder  of  the  baseline  data  analysis, 
and  the  post-implementation  plan  and  data  collection  were 
conducted  by  Arthur  D.  Little,  Inc.  This  resulted  in  some 
difficulty  in  interpretation  of  the  baseline  data;  for 
example,  it  was  not  clear  whether  the  total  times  estimated 
for  information  handling  and  other  activities  in  the  base¬ 
line  period  (Section  B)  referred  to  all  shifts  and  days  of 
the  week,  or  only  for  the  weekday  shifts  (when  the  work 
sampling  data  collection  was  mainly  carried  out) . 

•  Although  as  indicated  in  Chapter  I,  total  laboratory 
workload  and  staffing  has  remained  relatively  stable 
between  the  baseline  and  post-implementation  study  periods, 
staff  have  been  reassigned  among  laboratory  sections. 

Staffing  of  Chemistry  and  Microbiology  decreased,  while 
staffing  in  Hematology  increased.  The  differences  in 
staffing  levels,  however,  are  subject  to  uncertainty 
because  of  difficulties  in  interpretation  of  the  estimates 
of  staffing  levels  during  the  baseline  sampling  period. 
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•  The  Hematology  section  obtained  a  new  Coulter  S+ 
instrument  between  the  baseline  and  post-implementation 
study  periods.  The  new  instrument  has  a  higher  through¬ 
put  rate  than  the  former  Coulter  instrument  in  the 
laboratory. 

Despite  these  complications,  the  results  obtained  in  the  baseline 
and  post-implementation  studies  appear  sufficiently  consistent  that 
conclusions  can  be  drawn,  albeit  with  some  degree  of  uncertainty. 

B.  PERSONNEL  TIME  DEVOTED  TO  INFORMATION  HANDLING 
1.  Introduction 

In  the  baseline  study,  time  spent  by  laboratory  personnel  in 
information  handling  activities  within  the  laboratory  was  determined 
by  an  extensive  work  sampling  program  conducted  in  the  three  major 
laboratory  sections:  Chemistry,  Hematology,  and  Microbiology. 

Thirteen  information  handling  activities  were  defined  for  work 
sampling  purposes: 

•  verify  and  correct  result  request  slips; 

•  accession  specimens; 

•  prepare  worksheets; 

9  perform  test  calculations; 

•  record  test  results; 

•  transcribe  test  results; 

•  report  test  results; 

•  distribute  test  results; 

•  file  test  results; 

•  respond  to  inquiries  on  test  status; 

•  compile  workload  statistics; 

•  maintaining  quality  control  information;  and 

•  process  incoming  and  outgoing  specimens. 

Four  additional  activities  were  utilized  to  measure  time  devoted 
to  activities  other  than  information  handling: 

•  process  test  results  (perform  tests); 

•  other  productive  work; 

•  other  non-productive  work  (socializing,  nonbusiness 
use  of  telephone);  and 

•  away  from  area. 
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The  data  were  collected  mainly  during  the  daytime  shift  (primarily 
0700  to  1600)  and  on  weekdays,  during  an  eight-week  period. 

In  order  to  compare  the  time  spent  by  laboratory  personnel  on 
information  handling  activities  after  the  installation  of  the  TRILAB 
system,  the  work  sampling  program  was  repeated  during  the  post¬ 
implementation  study.  Seven  additional  activities  were  defined  to 
account  for  computer-related  activities: 

•  maintain  computer  system; 

•  accession  specimens; 

•  prepare  worksheets; 

•  results  entry; 

•  results  retrieval; 

•  results  review;  and 

•  edit  files. 

The  definition  of  the  work  sampling  activities  and  the  data  collection 
forms  are  provided  in  Appendix  A. 

In  order  to  obtain  data  comparable  to  that  collected  in  the 
baseline  period,  work  sampling  was  performed  during  the  day  shift, 
from  0730  to  1600,  on  weekdays,  for  a  10-day  period  between  October 
8-22,  1982.  Data  collectors  were  each  trained  in  activities  of  two 
laboratory  sections  and  traded  areas  for  part  of  each  day.  Quality 
control  of  the  data  collection  was  monitored  by  frequent  checking  by 
the  data  collection  supervisor. 

2.  Baseline  Results 

The  results  of  the  baseline  work  sampling  are  summarized  in  Table 

2 

3.  The  detailed  results  are  presented  in  Tables  4  ,5  and  6.“  Approxi¬ 
mately  52,000  observations  were  made  (21,000  in  Chemistry, 16,000  in 
Hematology,  and  14,500  in  Microbiology). 

Approximately  28  percent  of  laboratory  staff  time  was  devoted  to 
information  handling  activities  during  the  sampled  periods  (Table  3), 
estimated  to  account  for  a  total  of  410  manhours  per  week.  The  ANSER 
report  does  not  indicate  how  the  estimates  of  weekly  hours  per 
activity  were  calculated,  nor  does  it  provide  staffing  levels  by 
shift. 
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TABLE  4 


SUMMARY  OF  BASELINE  WORK  SAMPLING  RESULTS  FOR  THE  CHEMISTRY  SECTION 
OF  THE  CLINICAL  LABORATORY  AT  NRMC  OAKLAND 


Activity  Measured 

Number  of 
Observations 

Percent  of 
Total 

Time  Spent 

Manhours  Spent 
(per  week) 

Verify  and  correct 
result  request  slips 

95 

0.4 

2.9 

Accession  specimens 

2705 

12.8 

81.7 

Prepare  worksheets 

360 

1.7 

10.9 

Process  test  results 

4191 

19.8 

126.5 

Perform  test  calcula¬ 
tions 

146 

0.7 

4.4 

Record  test  results 

361 

1.7 

10.9 

Transcribe  test  results 

756 

3.6 

22.8 

Report  test  results 

124 

0.6 

3.7 

Distribute  test  results 

386 

1.8 

11.7 

File  test  results 

218 

1.0 

6.6 

Respond  to  inquiries 
on  test  status 

394 

1.9 

11.9 

Compile  workload 
statistics 

611 

2.9 

18.4 

Maintain  quality 
control  information 

1227 

5.8 

37.0 

Process  incoming  and 
outgoing  specimens 

514 

2.4 

15.5 

Other  productive  work 

2556 

12.1 

77.2 

Nonproductive  time 

1038 

4.9 

31.3 

Away  from  area 

5489 

25.9 

165.7 

Maintain  computer 
system 

„ 

21171 

100.0% 

639.1  TOTAL 

Source:  Analytic  Services,  Inc.  (ANSER),  "Progress  Report,  Period  X  Data 
Analysis  for  the  Evaluation  of  the  Tri-Service  Laboratory  Initial  Capa¬ 
bility  System  (TRILAB)  at  the  Naval  Regional  Medical  Center  Oakland, 
California."  Contract  No.  MDA  903-78-C-0085 .  Report  to  the  Tri-Service 
Medical  Information  Systems  Program  Office,  Bethesda,  MD,  January  1981. 
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TABLE  5 


SUMMARY  OF  BASELINE  WORK  SAMPLING  RESULTS  FOR  THE  HEMATOLOGY  SECTION 
OF  THE  CLINICAL  LABORATORY  AT  NRMC  OAKLAND 

Activitv  Measured 

Number  of 
Observations 

Percent  of 
Total 

Time  Spent 

Manhours  Spent 
(per  week) 

Verify  and  correct 
result  request  slips 

58 

0.4 

1.4 

Accession  specimens 

1077 

6.7 

25.4 

Prepare  worksheets 

73 

0.5 

1.7 

Process  test  results 

4829 

30.1 

113.7 

Perform  test  calcu¬ 
lations 

31 

0.2 

0.7 

Record  test  results 

159 

1.0 

3.7 

Transcribe  test  results 

466 

2.9 

11.0 

Report  test  results 

111 

0.7 

2.6 

Distribute  test  results 

156 

1.0 

3.7 

File  test  results 

119 

0.7 

2.8 

Respond  to  inquiries 
on  test  status 

286 

1.8 

6.7 

Compile  workload 
statistics 

142 

0.9 

3.3 

Maintain  quality 
control  information 

189 

1.2 

4.4 

Process  incoming  and 
outgoing  specimens 

24 

0.1 

0.6 

Other  productive  work 

1435 

8.9 

33.8 

Nonproductive  time 

287 

1.8 

6.8 

Away  from  area 

6621 

41.2 

155.9 

Maintain  computer 
system 

16063 

100. i%a 

378.2  TOTAL 

Source:  Analytic  Services,  Inc.  (ANSER) ,  "Progress  Report,  Period  X  Data 
Analysis  for  the  Evaluation  of  the  Tri-Service  Laboratory  Initial 
Capability  System  (TRILAB)  at  the  Naval  Regional  Medical  Center,  Oakland, 
California."  Contract  No.  MDA  903-78-C-0085 .  Report  to  the  Tri-Service 
Medical  Information  Systems  Program  Office,  Betiiesda,  MD,  January  1981. 

aDue  to  rounding. 
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TABLE  6 


SUMMARY  OF  BASELINE  WORK  SAMPLING  RESULTS  FOR  THE  MICROBIOLOGY  SECTION 
OF  THE  CLINICAL  LABORATORY  AT  NRMC  OAKLAND 


Activity  Measured 

Number  of 
Observations 

Percent  of 
Total 

Time  Spent 

Manhours  Spent 
(per  week) 

Verify  and  correct 
result  request  slips 

40 

0.3 

1.3 

Accession  specimens 

1530 

10.5 

48.2 

Prepare  worksheets 

50 

0.3 

1.6 

Process  test  results 

4230 

29.1 

133.4 

Perform  test  calcula¬ 
tions 

10 

0.1 

0.3 

Record  test  results 

348 

2.4 

11.0 

Transcribe  test  results 

442 

3.0 

13.9 

Report  test  results 

145 

1.0 

4.6 

Distribute  test  results 

123 

0.8 

3.9 

File  test  results 

118 

0.8 

3.7 

Respond  to  inquiries 
on  test  status 

247 

1.7 

7.8 

Compile  workload 
statistics 

51 

0.4 

1.6 

Maintain  quality 
control  information 

128 

0.9 

4.0 

Process  incoming  and 
outgoing  specimens 

52 

0.4 

1.6 

Other  productive  work 

1600 

11.0 

50.5 

Nonproductive  time 

502 

3.5 

15.8 

Away  from  area 

4917 

33.8 

155.1 

Maintain  computer 
system 

— 

— 

— 

14533 

100.0% 

458.3  TOTAL 

Source:  Analytic  Services,  Inc.  (ANSER) ,  "Progress  Report.  Period  X 
Data  Analysis  for  the  Evaluation  of  the  Tri-Service  Laboratory  Initial 
Capability  System  (TRILAB)  at  the  Naval  Regional  Medical  Center  Oakland, 
California."  Contract  No.  MDA  903-78-C-0085.  Report  to  the  Tri-Service 
Medical  Information  Systems  Program  Office,  Bethesda,  MD,  January  1981. 
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Review  of  the  raw  data  work  sampling  sheets  indicated  that  staff 
work  sampled  included  the  technicians  assigned  to  the  three 
departments,  including  trainees,  and  excluded  supervisors,  officers, 
and  secretaries.  The  hours  per  week  reported  in  Table  3  are 
equivalent  to  average  staffing  of  37,  16,  9.5  and  11.5  for  Chemistry, 
Hematology  and  Microbiology,  respectively — assuming  40  hours  per  week 
per  FTE.  It  is  not  clear  whether  this  staff  was  for  all  shifts  and 
days  of  the  week,  or  for  the  weekday  shifts  only.  The  sections 
demonstrated  a  wide  variance  in  proportion  of  information  handling 
time:  37  percent  in  Chemistry,  compared  with  20  percent  in  Hematology 
and  Microbiology.  Conversely,  about  20  percent  of  staff  tine  in 
Chemistry  was  devoted  to  actual  test  processing,  compared  with  30 
percent  for  Hematology  and  Microbiology. 

In  the  Chemistry  Section,  26  percent  of  staff  time  was  spent  away 
from  the  area,  reflecting  164.7  manhours  per  week.  This  time  included 
personal  time,  time  when  the  laboratory  was  being  cleaned,  and  filing 
time  in  the  reception  area.  Data  collected  in  the  Microbiology 
section  showed  staff  away  from  their  area  34  percent  of  the  time.  A 
portion  of  this  was  due  to  staff  being  required  to  spend  time  in  the 
refrigerated  media  room,  which  was  located  on  another  floor.  Hema¬ 
tology  staff  were  away  from  the  area  a  substantially  greater  propor¬ 
tion  of  time  (41.2  percent).  This  was  attributed  to  the  time  spent  by 
Hematology  staff  in  assisting  in  the  phlebotomy  and  reception  areas, 
which  were  physically  near  the  Hematology  laboratory. 

Another  activity  which  required  a  proportionately  large 
percentage  of  time  was  the  processing  of  test  results.  This  was 
evident  in  all  three  sections  of  the  laboratory — Hematology:  30.1 
percent  of  time  or  113.7  manhours /week;  Microbiology:  29.1  percent  of 
time  or  133.4  manhours/week;  and  Chemistry:  19.8  percent  of  time  or 
126.5  manhours/week.  Accessioning  specimens  also  required  a 
proportionately  large  amount  of  time. 

Finally,  a  significant  amount  of  time  in  all  three  sections  was 
spent  transcribing  test  results — Chemistry:  3.6  percent  of  cime  or 
22.8  manhours/week;  Hematology:  2.9  percent  of  time  or  11.0  manhours,' 
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week;  and  Microbiology:  3  percent  of  time  or  1.39  manhours /'week.  In 
addition,  data  collected  in  the  Chemistry  section  of  the  laboratory 
indicated  significant  time  was  devoted  to  compiling  workload  statis¬ 
tics  and  maintaining  quality  control  information — 18.4  and  37.0 
manhours,  respectively.  The  time  spent  maintaining  quality  control 
information  reflected  the  assignment  of  100  percent  of  one  staff 
member's  time  performing  only  workload  calculations  and  quality 
control  analysis. 

Further  interpretation  of  these  data  and  translation  into  a  form 
suitable  for  comparison  with  data  collected  in  the  post-implementation 
study  was  made  difficult  by  the  lack  of  data  on  staffing  and  workload 
processed  by  the  three  sections  during  the  eight-week  sampling  period 
(i.e.,  day  shift,  weekdays). 

3.  Post-Implementation  Results 

The  results  of  the  post-implementation  work  sampling  are 
summarized  in  Table  7.  The  detailed  results  for  each  of  the  sections 
(Chemistry,  Hematology,  and  Microbiology)  are  presented  in  Tables  8,  9 
and  10.  The  only  planned  difference  between  the  work  sampling  in  the 
two  study  periods  was  that  the  Nuclear  Medicine  section  was  not 
included  in  the  post-implementation  work  sampling,  because  the  TRILAB 
system  was  only  partially  implemented  there  and  section  activities 
were,  therefore,  not  considered  representative  of  those  when  the 
system  will  be  fully  implemented  in  that  section. 

Approximately  29,000  observations  were  obtained,  approximately 
one  third  in  each  of  the  major  sections.  Because  the  computer  was 
down  for  virtually  one  complete  day  during  the  data  collection  period, 
however,  the  results  for  that  day  were  omitted  from  subsequent  calcu¬ 
lations  as  being  unrepresentative  of  normal  operations  with  the  TRILAB 
system.  This  resulted  in  some  26,000  observations  considered  useful 
(Table  7). 

Approximately  24.6  percent  of  laboratory  staff  time  was  devoted 
to  information  handling  activities  during  the  sampled  period  (Table 
7).  As  in  the  baseline  period,  percent  of  time  devoted  to  information 
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TABLE  7 


SUMMARY  OF  POST-IMPLEMENTATION  WORK  SAMPLING  RESULTS 
TRILAB  EVALUATION  IN  THE  CLINICAL  LABORATORY  AT  NRMC  OAKLAND 


Section 

Number  of 
Observations'1 

Total 

Hours /Week 

Time  Spent  in 
Information  Handling 

Percent 

Hours /Week 

Chemistry 

8,441 

420 

38.5 

161.  7 

Hematology 

9,562 

480 

15.4 

73.9 

Microbiology 

8.099 

400 

21.0 

84.0 

TOTAL 

26,102 

1,300 

24.6 

319.6 

a 


Excludes  one  day's  observations  when  computer  was  down. 
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TABLE  8 


SUMMARY  OF  POST- IMPLEMENTATION  WORK  SAMPLING  RESULTS  FOR  THE  CHEMISTRY  SECTION 
TRILAB  EVALUATION  IN  THE  CLINICAL  LABORATORY  AT  NRMC  OAKLAND 


Activity  Measured 

Number  of 
Observations 

Percent 
of  Total 
Time  Spent 

Manhours 
per  Week 

1. 

Verify  and  Correct  Request  Slips 

1 

0 

0 

Accession  Specimens 

854 

10. 1 

42.  4 

3. 

Prepare  Worksheets 

154 

1.8 

7.6 

4. 

Process  Test  Results 

1,758 

87.4 

5. 

Perform  Test  Calculations 

36 

/  0.4 

1.  7 

6. 

Record  Test  Results 

309  / 

89^/ 

id 

*  3. 7 

15.5 

7. 

Transcribe  Test  Results 

1. 1 

4.6 

8. 

Report  Test  Results 

0.1 

0.4 

9. 

Distribute  Test  Results 

2 

0 

0 

10. 

File  Test  Results 

34 

0.4 

1.7 

11. 

Respond  to  Inquiries 

91 

1.1 

4.6 

12. 

Compile  Workload  Statistics 

0 

0 

0 

13. 

Log,  Calculate  and  Report  Quality  Control 

107 

1.3 

5.5 

14. 

Administrative  Handling 

15 

0.2 

0.8 

15. 

Other — Productive 

1,814 

21.5 

90.  3 

16. 

Other — Nonproductive 

170 

2.0 

8.4 

17. 

Away  from  Area 

1,450 

17.2 

72.2 

18. 

Maintain  Computer  System 

23 

0.3 

1.  3 

19. 

Accession 

904 

10.7 

44.9 

20. 

Prepare  Worksheets 

72 

0.9 

3.3 

21. 

Results  Entry 

472 

5.6 

23.5 

22. 

Results  Retrieval 

33 

0.4 

1.  7 

23. 

Results  Review 

32 

0.4 

1.  7 

24. 

Edit  File 

11 

0.1 

0.4 

TOTAL 

8,441 

100. 1 

410.4 
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TABLE  9 

SUMMARY  OF  POST- IMPLEMENTATION  WORK  SAMPLING  RESULTS  FOR  THE  HEMATOLOGY  SECTION 
TRILAB  EVALUATION  IN  THE  CLINICAL  LABORATORY  AT  NRMC  OAKLAND 


Percent 


m 

Activity  Measured 

Number  of 
Observations 

of  Total 
Time  Spent 

Manhours 
per  Week 

1. 

Verify  and  Correct  Request  Slips 

0 

0 

0 

V, 

2. 

Accession  Specimens 

25 

0.3 

1.4 

3. 

Prepare  Worksheets 

8 

0. 1 

0.5 

4. 

Process  Test  Results 

2,502 

26.  2 

125.8 

5. 

Perform  Test  Calculations 

12 

0.1 

0.5 

6. 

Record  Test  Results 

9 

0. 1 

0.5 

a 

7. 

Transcribe  Test  Results 

11 

0.1 

0.5 

>: 

8. 

Report  Test  Results 

15 

0.2 

1.0 

A 

.v 

9. 

Distribute  Test  Results 

2 

0 

0 

10. 

File  Test  Results 

32 

0.3 

1.4 

| 

11. 

Respond  to  Inquiries 

17 

0.2 

1.0 

12. 

Compile  Workload  Statistics 

1 

0 

0 

/*, 

rs 

13. 

Log,  Calculate  and  Report  Quality  Control 

79 

0.8 

3.  8 

■■* 

14. 

Administrative  Handling 

36 

0.4 

1.9 

9 

15. 

Other — Productive 

2,123 

22.2 

106.6 

16. 

Other — Nonproductive 

401 

4.2 

20.  2 

V. 

17. 

Away  from  Area 

3,060 

32.0 

153.6 

V, 

V. 

18. 

Maintain  Computer  System 

67 

0.7 

3.  4 

19. 

Accession 

551 

5.8 

27.8 

'.s 

*  J 

20. 

Prepare  Worksheets 

22 

0.2 

1.0 

21. 

Results  Entry 

540 

5.6 

26.9 

$ 

22. 

Results  Retrieval 

21 

0.2 

1.0 

23. 

Results  Review 

4 

0 

0 

24. 

Edit  File 

24 

0.  3 

1.  4 

V 

TOTAL 

9,562 

100.0 

480.2 
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TABLE  10 

SUMMARY  OF  POST- IMPLEMENTATION  WORK  SAMPLING  RESULTS  FOR  THE  MICROBIOLOGY  SECTION 
TRILAB  EVALUATION  IN  THE  CLINICAL  LABORATORY  AT  NRMC  OAKLAND 


Activity  Measured 
Verify  and  Correct  Request  Slips 
Accession  Specimens 
Prepare  Worksheets 
Process  Test  Results 
Perform  Test  Calculations 
Record  Test  Results 
Transcribe  Test  Results 
Report  Test  Results 
Distribute  Test  Results 
File  Test  Results 
Respond  to  Inquiries 
Compile  Workload  Statistics 
Log,  Calculate  and  Report  Quality  Control 
Administrative  Handling 
Other— P  roduct  ive 
Other — Nonproductive 
Away  from  Area 
Maintain  Computer  System 
Accession 
Prepare  Worksheets 
Results  Entry 
Results  Retrieval 
Results  Review 
Edit  File 
TOTAL 


Number  of 
Observations 


2,749 


Percent 
of  Total 
Time  Spent 


Manhours 
er  Week 
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handling  varied  widely  among  sections:  from  15.4  percent  in 
Hematology  to  38.5  percent  in  Chemistry.  Based  on  average  day-shift 
staffing  during  the  post-implementation  data  collection  period  of  32.5 
(10.5  in  Chemistry,  12  in  Hematology,  and  10  in  Microbiology),  and 
assuming  40  hours  per  week  per  FTE,  or  1300  hours  per  week  total, 
information  handling  activities  accounted  fjr  about  320  hours  per 
week.  Approximately  60.5  percent  of  the  time  spent  on  information 
handling  activities,  or  194  hours  per  week,  was  spent  on 
computer-related  activities. 

The  post-implementation  staffing  compares  with  the  average 
staffing  of  37  (16,  9,  5,  and  11.5)  for  Chemistry,  Hematology  and 
Microbiology  estimated  for  the  baseline  period,  or  35  (14,  9,5,  and 
11.5)  if  the  two  Nuclear  Medicine  staff  are  omitted.  Thus  overall 
day-shift  staffing  of  the  laboratory  was  reduced  by  2.5  FTE,  with 
decreases  of  3.5  in  Chemistry  and  1.5  in  Microbiology,  and  an  increase 
cf  2.5  in  Hematology.  These  differences  in  staffing  levels,  however, 
are  subject  to  error  because  of  difficulties  in  interpretation  of  the 
estimates  of  staffing  levels  during  the  baseline  sampling  period  (see 
Volume  II,  page  15,  of  the  Baseline  Report  (3)). 

In  the  Chemistry  section,  about  17  percent  of  staff  time  was 
spent  away  from  the  area,  equivalent  to  72  hours  per  week.  This  time 
included  personal  time,  time  when  the  laboratory  was  being  cleaned, 
and  time  when  staff  were  required  to  be  outside  their  section.  Data 
collected  for  the  Hematology  section  indicated  that  staff  were  away 
from  the  area  32  percent  of  the  time,  equivalent  to  154  hours  per 
week;  as  in  the  baseline  period,  this  large  percent  of  time  is 
attributed  partly  to  the  time  spent  by  Hematology  staff  in  assisting 
in  the  phlebotomy  and  reception  areas,  which  are  physically  near  the 
Hematology  Laboratory  and  under  its  jurisdiction.  Microbiology  staff 
spent  about  25  percent  of  the  time  away  from  the  area,  equivalent  to 
102  hours  per  week;  a  portion  of  this  time  was  due  to  staff  being 
required  to  spend  time  in  the  refrigerated  media  room,  which  was 
located  on  another  floor. 
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Another  activity  that  required  a  proportionately  large  percentage 
of  time  was  the  processing  of  test  results.  This  was  evident  in  all 
three  sections  of  the  laboratory:  Microbiology:  33.9  percent, 
equivalent  to  135.6  hours  per  week;  Hematology:  26.2  percent, 
equivalent  to  126  hours  per  week;  Chemistry:  20.8  percent,  equivalent 
to  87.4  hours  per  week. 

4.  Comparison  of  Baseline  and  Post-Implementation  Results 

Table  11  compares  the  time  devoted  to  information  handling 
activities  in  the  baseline  and  post-implementation  periods,  in  terms 
of  both  percent  of  time  and  estimated  hours  per  week.  The  comparisons 
of  percentage  of  time  are  considered  more  accurate  than  the 
comparisons  of  estimated  hours  per  week,  due  to  difficulties  in 
interpretation  of  the  estimates  of  weekly  hours  and  staffing  levels  in 
the  three  sections  during  the  baseline  sampling  period.  (Baseline 
hours  per  week  for  Chemistry  have  been  adjusted  for  comparison 
purposes  to  reflect  the  fact  that  the  Nuclear  Medicine  section  was  not 
included  in  post-implementation  sampling;  Chemistry  staff  hours  were 
reduced  by  12.5  percent  to  make  the  results  comparable.) 

Overall,  the  percentage  of  time  devoted  to  information  handling 
activities  was  2.6  percent  lower  in  the  post-implementation  than  in 
the  baseline  period;  this  difference  was  statistically  significant. 

Statistically  significant  (at  a  95  percent  confidence  level) 
reductions  in  the  percentage  of  time  devoted  to  information  handling 
activities  were  observed  in  Hematology  arid  Microbiology  sections  (2.7 
percent  and  1.6  percent,  respectively).  An  increase  of  1.2  percent  of 
time  devoted  to  information  handling  activities  was  observed  for 
Chemistry.  This  increase,  however,  was  not  statistically  significant. 
The  overall  reduction  in  hours  per  week  devoted  to  information 
handling  between  the  two  study  periods  was  about  60  hours  per  week. 
Part  of  this  reduction,  however,  was  due  to  the  difference  in 
staffing.  At  the  staffing  level  during  the  post-implementation  study, 
a  reduction  of  1.6  percent  was  equivalent  to  a  reduction  of  33.8  hours 
per  week  devoted  to  information  handling  activities,  or  slightly  less 
than  one  FTE. 
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TABLE  11 

COMPARISON  OF  BASELINE  AND  POST- IMPLEMENTATION  INFORMATION  HANDLING  TIMES 
TRILAB  EVALUATION  IN  THE  CLINICAL  LABORATORY  AT  NRMC  OAKLAND 


_ _ Percent  of  Time _ 

Base-  Post-  Differ- 

line  Implementation  ence 


_ Hours  per  Week _ 

Base-  Post-  Differ- 

line  Implementation  ence 


Chemistry 

37.  3% 

38.5% 

1.2% 

208. 6a 

161.7 

-46.9 

Hematology 

18. 1 

15.4 

-2.  7b 

68.0 

73.9 

5.9 

Microbiology 

22.6 

21.  0 

-1.6b 

103.5 

84.0 

-19.5 

All 

27. 2a 

24.6 

-2. 6b 

380.1 

319.6 

-60.5 

Adjusted  for  Nuclear  Medicine  staff. 

^Difference  is  statistically  significant  at  55^  confidence  level 
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Table  12  compares  the  percentage  of  time  devoted  to  several 
selected  activities  in  the  two  study  periods.  Overall,  time  devoted 
to  processing  of  test  results  increased  by  approximately  1.2  percent. 
As  might  be  expected,  time  devoted  to  transcription  and  recording  of 
test  results  was  reduced  by  about  0.9  percent,  equivalent  to  about  16 
hours  per  week.  Time  devoted  to  compilation  of  workload  statistics, 
which  accounted  for  1.5  percent,  or  21  hours  per  week  of  time  in  the 
baseline  period,  was  eliminated  in  the  post-implementation  period 
because  the  computer  system  assumed  this  function.  Time  devoted  to 
quality  control  logging,  calculation,  and  updating  was  reduced  from 
2.9  percent  to  0.7  percent  of  total  time,  or  by  31  hours  per  week. 

Time  spent  away  from  the  area  was  25.2  percent  in  the 
post-implementation  period  compared  with  32.7  percent  in  the  baseline. 
It  is  not  clear  whether  this  was  due  to  differences  in  sampling 
methodology,  or  whether  the  staff  did  spend  more  time  in  the 
laboratory  sections,  possibly  as  a  result  of  reduction  in  available 
staff. 

5.  Conclusion 

It  is  concluded  that  time  devoted  to  information  handling 
activities  in  the  post-implementation  period  was  approximately  2.6 
percent  lower  than  in  the  baseline.  Based  on  current  staffing,  this 
was  equivalent  to  a  net  reduction  of  34  hours  per  week  (day  shift, 
Monday  to  Friday)  devoted  to  information  handling  activities  in  the 
Chemistry,  Hematology  and  Microbiology  sections.  As  expected,  there 
were  reductions  in  time  devoted  to  transcription  and  recording  of  test 
results,  compilation  of  workload  statistics,  and  for  quality  control 
reporting. 

C.  SERVICE  PERFORMANCE 
1.  Turnaround  Time 

a.  Baseline  Results 

During  the  baseline  data  collection,  information  was  obtained  on 
"process  times"  within  the  laboratory;  that  is,  the  time  between  when 
the  request  slip  was  received  in  the  laboratory  and  the  time:  (a)  the 
result  was  telephone  back  to  the  requester,  in  the  case  of  emergent/ 
urgent  requests,  or  (b)  the  time  the  completed  slip  was  placed  in  the 
distribution  box  to  be  picked  up  by  staff  from  the  wards  or  clinics, 
for  routine  requests. 

A  34 
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The  raw  data  obtained  during  the  baseline  data  collection  were 
reviewed.  Some  of  the  data  was  unusable  due  to  lack  of  identification 
of  data  sheets,  obvious  errors  (e.g.,  receipt  dates  being  later  than 
dates  of  results  report) ,  and  errors  in  the  calculation  of  process 
times.  After  errors  were  corrected,  questionable  data  eliminated  and 
data  points  excluded  that  were  more  than  six  standard  deviations  from 
the  mean,  a  total  of  412  usable  observations  remained  for  analysis. 
The  results  are  summarized  in  Table  13.  The  majority  of  the 
observations  (250)  were  made  for  routine  results  for  Hematology  tests, 
with  the  remainder  made  for  routine  Chemistry,  STAT  Chemistry,  STAT 
Hematology,  and  Microbiology  tests. 

In  the  case  of  both  Chemistry  and  Hematology  STAT  requests, 
results  were  phoned  back  to  the  requester  an  average  of  1.25  hours 
after  receipt  of  the  slip  by  the  laboratory  (SD=0.7  hours).  Routine 
Chemistry  results  were  available  in  the  distribution  box  an  average  of 
16  hours  after  receipt  by  the  laboratory,  whereas  routine  Hematology 
results  were  available  an  average  of  4.7  hours  after  receipt.  The 
average  for  availability  of  Microbiology  results  was  39  hours  after 
receipt.  The  distribution  of  process  turnaround  times  was  quite  broad 
for  the  Chemistry  and  Hematology  routine  results,  with  a  coefficient 
of  variation  (standard  deviation/mean)  greater  than  1,  whereas  the 
distributions  for  STAT  results  and  for  Microbiology  results  were  much 
narrower,  with  a  coefficient  of  variation  of  0.55  or  less. 

b.  Post-Implementation  Results 

Because  of  major  differences  in  the  way  results  were  returned 
from  the  laboratory  to  the  clinics  and  wards  in  the 
post-implementation  period,  the  method  by  which  the  process  times  were 
measured  was  different: 

STAT  Turnaround  Time.  The  TRILAB  computer  system  recorded  both 
the  time  (date  and  time  of  day)  "received"  (the  accession  time),  and 
the  time  that  the  result  was  verified  and  transmitted  to  the 
requesting  ward  or  clinic  (a  "beep"  was  produced  at  the  appropriate 
clinic  or  ward  when  the  result  was  available).  Consequently  these 
times  were  obtained  via  terminal  lcok-up  of  appropriate  test  records. 
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TABLE  13 


BASELINE  PROCESS  TURNAROUND  TIMES 
FOR  SELECTED  TESTS 

IN  THE  CLINICAL  LABORATORY  AT  NRMC  OAKLAND 


Sample  Size 


Mean 

rs)  S.D.  (Hours) 


Chemistry  Routine  52 
Hematology  Routine  254 
Chemistry  ST AT  39 
Hematology  STAT  64 
Microbiology  16 


16.10 


18.56 


39.10 


13.91 


Coefficient 
of  Variance 
(o/x) 


aAfter  deleting  thirteen  observations  greater  than  +  6  times  the  standard 
deviation  from  the  mean. 
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Routine  Results  Turnaround  Time.  The  time  that  routine  results 
were  available  to  requesters  via  terminal  inquiry  was  similarly 
determined,  using  terminal  look-up. 

Interim  Report  Results  Turnaround  Time.  The  times  that  the 
interim  reports  were  available  was  obtained  by  copying  the  print  end 
times  for  those  requisition  in  the  sample  for  which  interim  reports 
are  produced.  These  include  only  the  inpatient  floors  with  surgical 
patients  (fourth  floor,  6  East,  6  West,  6  North,  7  East,  7  West,  7 
North,  8  East,  8  West,  and  8  South).  Thus  those  requisitions  for 
which  an  interim  report  time  was  available  constituted  a  subset  of  the 
original  samples. 

Daily  Summary  Reports  Process  Time.  A  similar  procedure  was 
followed  to  obtain  the  print  end  times  for  the  daily  summary  reports. 

Report  Pickup  Time.  The  time  that  the  daily  reports  were  picked 
up  for  distribution  to  wards  and  clinics  was  obtained  from  the  logs 
kept  in  the  data  processing  department. 

Table  14  summarizes  the  sample  sizes  and  averages,  medians,  and 
standard  deviations  of  the  process  times  obtained.  Five  process  times 
are  shown: 

(1)  CRT  Process  Time  (first  results  available):  the  elapsed 
time  between  accessioning  of  a  request  by  the  laboratory 
department  and  availability  of  the  first  results  via 
terminal . 

(2)  CRT  Process  Tine  (final  results  available):  the  time 
between  accessioning  of  the  requisition  and  availability 
of  all  test  results  via  terminal  inquiry. 

(3)  Interim  Report  Process  Time:  the  time  between  requisition 
accessioning  and  availability  of  the  "daily  interim  hard¬ 
copy  report"  for  pickup  in  the  Data  Processing  Department. 

(4)  Daily  Report  Process  Time:  the  time  between  accessioning 
and  availability  of  the  daily  reports  for  pickup  by  clinic/ 
ward  personnel. 

(5)  Final  Report  Pickup  Time:  the  time  between  accessioning 
and  the  time  the  daily  report  was  picked  up. 
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Samples  have  been  adjusted  to  exclude  +  4  times  the  standard  deviation 
(reduced  Chemistry  STAT  te3ts  by  2  and  Hematology  STAT  and  routine  tests  by  1). 


A  total  of  493  observations  was  obtained  (after  excluding  outlier 
data  points  more  than  four  standard  deviations  from  the  mean) .  The 
number  of  observations  of  each  test  category  was  chosen  to  be 
consistent  with  the  sample  size  and  variability  of  observations  in  the 
baseline  period.  As  in  the  baseline  period.  Hematology  routine 
observations  constituted  the  largest  subsample,  with  the  remainder  for 
routine  Chemistry,  STAT/urgent  Hematology,  STAT/urgent  Chemistry,  and 
Microbiology  tests. 

In  the  case  of  STAT/urgent  tests,  first  results  were  available  on 
average  1.5  hours  after  accessioning  for  Chemistry  tests,  and  0.85 
hours  after  accessioning  for  Hematology  tests.  Final  results  (which 
generally  included  non-STAT/urgent  tests  on  the  requisition)  were 
available  via  CRT  inquiry  3.5  hours  after  requisition  for  Chemistry, 
and  2.2  hours  for  Hematology. 

In  the  case  of  the  routine  request  requisitions,  results  were 
available  for  Chemistry  tests  on  average  24  hours  after  accessioning, 
2  hours  for  Hematology  tests,  and  60  hours  for  Microbiology  tests. 

Interim  reports  were  available  (for  patients  on  surgical  floors) 
approximately  10  hours  after  accessioning  for  Chemistry  and  Hematology 
STAT/urgent  tests,  22  hours  for  Chemistry  routine  tests,  9.5  hours  for 
Hematology  routine  tests,  and  64  hours  for  Microbiology  tests.  The 
daily  reports  were  available  approximately  18  hours  after  accessioning 
for  Chemistry  and  Hematology  STAT/urgent  tests,  39  hours  for  Chemistry 
routine  tests,  16  hours  for  Hematology  routine  tests,  and  74  hours  for 
Microbiology  tests.  The  average  pickup  time  for  the  reports  was  21-24 
hours  for  Hematology  and  Chemistry  STAT/urgent  tests,  54  hours  for 
Chemistry  routine  tests,  26  hours  for  Hematology  routine  tests,  and  94 
hours  for  Microbiology  tests. 

The  distribution  of  "CRT  process  times"  for  routine  Chemistry 
tests  was  quite  broad,  with  a  coefficient  of  variation  (standard 
deviation/mean)  of  about  1.4,  whereas  the  distributions  for  the  other 
test  type  categories  were  much  narrower,  with  coefficients  of 
variation  of  0.9  or  less. 
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c. 


Comparison  of  Baseline  and  Post-Implementation  Laboratory 

Process  Times 

Table  15  compares  process  times  observed  in  the  two  study 
periods.  In  the  case  of  the  STAT/urgent  tests,  the  post¬ 
implementation  averages  presented  are  for  the  "CRT  process  time" 
(first  results  available),  as  being  most  comparable  to  the  process 
time  measured  in  the  baseline  period  (time  when  results  were 
telephoned  back  to  the  requesting  units).  In  the  case  of  routine 
tests,  the  final  results  available  via  CRT  are  presented,  as  being 
most  comparable  to  process  times  measured  in  the  baseline  period 
(completed  results  slips  available  for  pickup) . 

The  results  suggest  that  process  times  were  reduced  in  the 
post-implementation  period  compared  to  the  baseline  period  for 
Hematology  STAT/urgent  test  (by  0.4  hours)  and  for  Hematology  routine 
tests  (by  2.6  hours);  these  differences  were  statistically  significant 
at  the  95  percent  confidence  level.  The  process  times  for  STAT/urgent 
and  routine  Chemistry  tests  increased  (by  0.23  hours  and  8.3  hours, 
respectively);  these  differences,  however,  were  not  statistically 
significant.  Average  process  times  for  Microbiology  tests  increased 
by  about  20  hours;  this  difference  was  statistically  significant. 

The  differences  observed  must  be  interpreted  with  caution  for  the 
following  reasons: 

•  In  the  case  of  routine  tests,  the  process  times  are  not 
entirely  comparable  because  in  the  baseline  period  the 
process  time  represents  the  time  when  the  requisition 
slip  was  available  for  pickup,  where  the  post-implementation 
process  time  represents  the  time  that  the  result  was  in 
fact  available  to  the  requester  (via  terminal  look-up) . 

The  time  between  availability  of  the  completed  requisition 
slip  and  pickup  or  availability  of  the  result  to  the 
requester  was  not  measured  in  the  baseline  study  at 
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TABLE  15 


COMPARISON  OF  BASELINE  AND  POST- IMPLEMENTATION  AVERAGE  PROCESS  TIMES 
TRILAB  EVALUATION  IN  THE  CLINICAL  LABORATORY  AT  NRMC  OAKLAND 


Mean  Turnaround  Time  (hours) 


STAT/Urgent 

Baseline 

Post-Implementation 

Difference 

F  Test 

Chemistry 

1.27a 

1.50b 

0.23 

2.6* 

Hematology 

1.25a 

rd 

m 

00 

-0.40* 

*  1.07 

Routine 

Chemistry 

16. 1C 

24.  4d 

8.3 

2.8* 

Hematology 

4.7C 

2.1d 

-2.6* 

12.  3* 

Microbiology 

39. 1C 

59. 6d 

20.5* 

3.5* 

g 

^Time  from  receipt  of  specimen  by  laboratory  to  telephoning  results. 
cTime  from  accessioning  to  availability  of  first  results  via  terminal. 

^Time  from  receipt  of  specimen  to  availability  of  results  3t  distribution  box. 

Time  from  accessioning  to  availability  of  final  results  via  terminal. 
*Statistically  significant  difference  at  955  confidence  level. 
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NRMC  Oakland.  At  DDEAMC,  however,  total  turnaround 
time  was  measured  in  addition  to  "process  time;"  the 
difference  varied  between  17  hours  and  20  hours,  on 
average.  Thus  routine  results  were  available  to  the 
provider  (via  terminal  inquiry)  considerably  sooner  in 
the  post-implementation  period  for  routine  Chemistry 
and  Hematology  tests,  and  probably  in  about  the  same 
time  for  Microbiology  tests. 

•  During  the  post-implementation  period,  the  tests  to  be 
sampled  in  each  test  type  category  were  chosen  by  using 
an  appropriate  "skip"  interval  between  tests,  in  order 
to  obtain  a  random  sample  representative  of  turnaround 
times.  It  is  not  clear  how  the  sample  for  the  baseline 
period  data  collection  was  chosen;  as  indicated  in  the 
baseline  report,  much  of  the  data  was  unusable  due  to 
lack  of  identification  of  data  sheets,  obvious  errors 
(e.g.,  receipt  dates  being  later  than  dates  of  results 
report)  and  errors  in  the  calculation  of  process  times, 
resulting  in  some  questionable  data.  In  fact,  there  is 
evidence  that  the  distributions  (spreads)  of  test  result 
process  times  are  not  the  same  in  the  two  periods,  as 
measured  by  the  statistical  F  Test.  As  indicated  in 
Table  15  the  distribution  of  test  results  in  the  two 
study  periods  are  significantly  different  for  each  test 
category,  except  for  Hematology  STAT/urgent  tests. 

•  Operating  conditions  in  the  laboratory  were  not  the  same 
in  the  two  study  periods.  The  Hematology  Laboratory  had 
obtained  a  new  Coulter  S+  instrument,  which  had  a  higher 
throughput  rate  than  the  previous  instrument  (the  Coulter 


* 

A  mini  baseline  evaluation  was  conducted  to  evaluate  TRILAB  at 
Eisenhower  AMC.  Complete  results  are  summarized  in  the  baseline 
report.  Volume  III  (3). 
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was  the  major  instrument  utilized  in  the  Hematology  section) . 
Staffing  in  the  Chemistry  section,  as  mentioned  earlier,  was 
somewhat  reduced  from  that  of  the  baseline  period.  Also, 
staff  were  preparing  for  the  Joint  Accreditation  visit, 
which  may  have  resulted  in  fewer  staff  being  available  for 
production  of  tests. 

•  The  apparent  increase  in  Microbiology  test  times  is 
difficult  to  explain,  except  for  the  possibility  that 
the  smaller  baseline  sample  (16  observations)  may  not  be 
representative  of  the  tests  carried  out  in  the  Microbiology 
section  (specific  tests  and  requesting  locations  were  not 
provided  in  the  baseline  data);  as  mentioned  above,  the 
distributions  of  process  times  were  significantly  different. 
Since  test  times  in  Microbiology  are  longer  and  vary  so 
much,  depending  on  type  of  specimen  and  results  (positive 
or  negative),  differences  in  the  type  of  tests  sampled  in 
the  baseline  and  post-implementation  periods  could  result 
in  considerably  different  test  times,  which  masked  any 
differences  in  reporting  times. 

d.  Conclusions 

Given  the  above  qualifications,  the  following  conclusions  may  be 
drawn: 

(1)  For  STAT/urgent  tests,  process  times — times  between 
receipt  of  requisition  and  transmission  of  test  results 
to  provider  locations  (via  telephone  in  the  baseline 
period  and  via  terminal  in  the  post-implementation 
period) — were  either  unchanged  (Chemistry)  or  reduced 
(Hematology) . 

(2)  For  routine  tests,  results  were  available  to  provider 
locations  (via  terminal)  sooner  in  the  post-implementation 
period  than  in  the  baseline  period  for  Chemistry  and 
Hematology,  and  in  about  the  same  time  period  or  sooner 
for  Microbiology.  Hard-copy  daily  reports  were  generally 
available  to  providers  later  than  the  completed  results 
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requisition  slips  were  in  the  baseline  period.  Interim 
hard-copy  reports  (for  the  surgical  floors)  were  available 
sooner  in  the  post-implementation  period  for  Hematology 
tests,  and  in  approximately  the  same  time  for  Chemistry 
and  Microbiology  tests. 

2.  Number  of  Telephone  Calls 

a.  Baseline  Results 

The  number  of  telephone  calls  received  by  the  laboratory  was 
monitored  over  a  period  of  seven  days  during  the  day  shift,  Monday 
through  Friday  in  the  baseline  study.  Calls  were  categorized  as: 

(1)  request  for  results  on  file; 

(2)  request  for  information  on  other  results; 

(3)  request  for  the  supervisor; 

(4)  request  for  technician;  or 

(5)  request  for  general  information. 

Table  16  summarizes  the  number  of  calls  received,  by  type  of 
call.  A  total  of  724  calls  were  received  during  the  total 
observations  period  of  2,879  minutes,  equivalent  to  724  x  480/2,879  or 
about  102  calls  received  per  (8-hour)  day.  Based  on  the  147,625 
patient  tests  reported  performed  during  October  of  1980,  the  average 
daily  test  load  was  4,762  tests  per  day  for  the  31  day  period. 
Assuming  the  number  of  telephone  calls  to  the  laboratory  was  similar 
on  weekdays  and  weekends,  this  represented  an  average  of  one  call  for 
every  46.5  tests  performed.  Sixty  percent  of  the  calls  were 
requesting  information  from  the  laboratory  with  regard  to  test 
results;  an  additional  12  percent  of  calls  were  for  filed  results. 

b.  Post-Implementation  Results 

The  number  of  telephone  calls  received  by  the  laboratory  was 
monitored  over  a  period  of  10  days  during  the  post-implementation 
study.  These  data  were  collected  during  the  day  shift,  Monday  through 
Friday,  during  the  period  October  8-22,  1982.  The  monitoring 

procedures  utilized  were  similar  to  those  in  the  baseline  period.  As 
in  che  baseline  period,  calls  were  categorized  as: 
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COMPARISON  OF  BASELINE  AND  POST- IMPLEMENTATION  PERIOD  TELEPHONE  CALL  RESULTS 
TRILAB  EVALUATION  IN  THE  CLINICAL  LABORATORY  AT  NRMC  OAKLAND 


Number  per  Eight-Hour  Day 


Ratio  of  Baseline  to 
Post-Implementation 


Normalized 
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(1)  calls  requesting  information  from  filed  results; 

(2)  calls  requesting  information  from  the  laboratory; 

(3)  calls  for  a  specific  supervisor; 

(4)  calls  for  a  technician,  and 

(5)  calls  for  general  information. 

Table  16  provides  a  comparison  of  the  number  of  calls  per  day 
received  by  the  laboratory  during  the  two  study  periods.  In  the 
post-implementation  period  study,  the  laboratory  received  an  average 
of  67.4  calls  per  day.  The  total  number  of  calls  received  per  day 
during  the  post-implementation  period  was,  therefore,  two-thirds  that 
received  during  the  baseline  period.  The  distribution  of  calls  by 
type  was  similar  to  that  received  during  the  baseline  period,  with  the 
majority  of  calls  (51  percent)  requesting  information  from  the 
laboratory  with  regard  to  test  results.  The  number  of  calls  received 
was  lor,er  in  each  category  of  call  request,  except  for  calls  to 
technicians,  which  doubled  from  9  calls  per  day  during  the  baseline 
period  to  about  18  calls  per  day  during  the  post-implementation 
period.  The  reason  for  the  increase  in  calls  in  this  category  is  not 
known. 

Based  on  the  170,150  patient  tests  reported  performed  during 
October  1982,  the  average  daily  test  load  was  5,489  tests  per  day  for 
the  31-day  period.  Assuming  the  number  of  telephone  calls  to  the 
laboratory  was  similar  on  weekdays  and  weekends,  this  represented  an 
average  of  one  call  for  every  81.4  tests  performed.  This  was  a 
reduction  of  43  percent  from  the  average  of  1  call  for  every  46.5 
tests  measured  during  the  baseline  period. 

It  should  be  noted  that  not  all  nursing  stations  or  clinics  had 
terminals  and  not  all  areas  of  the  laboratory  (viz.,  Nuclear  Medicine, 
Pathology  and  Blood  Bank)  were  on  the  system.  When  the  remaining 
areas  are  put  on  the  system,  and  if  additional  terminals  are  obtained 
for  provider  areas,  the  volume  of  telephone  calls  could  be  expected  to 
be  further  reduced. 
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c.  Conclusions 

The  volume  of  telephone  calls  to  the  laboratory  was  considerably 
reduced  by  implementation  of  the  TRILAB  system.  Normalized  to  study 
period  workloads,  the  volume  of  telephone  calls  in  the  post¬ 
implementation  period  was  almost  half  (57  percent)  that  in  the  base¬ 
line  study  period. 


D.  STAFF  PERCEPTIONS  OF  LABORATORY  SERVICES 


a.  Introduction 


Study  questionnaires  were  distributed  to  the  following  groups  of 
personnel  at  the  hospital  during  the  baseline  survey: 


No.  Sent 

No.  Returned 

Response  Rate 

1. 

All  medical  staff  (MDs 
and  dentists) 

276 

183 

66.2% 

2. 

All  nurses 

243 

146 

60.1 

3. 

Administrative,  corpsmen 
and  clerical  staff 

103 

57 

55.4 

4. 

Laboratory  staff 

73 

63 

83.3 

695 

449 

64.3 

The  purpose  of  the  questionnaires  was  to  determine  the  degree  of 
satisfaction  of  various  providers  and  staff  with  regard  to  their 
perceptions  of  laboratory  operations.  The  questionnaire  included  19 
questions  regarding  the  respondent's  perception  of  the  clinical 
laboratory  operations  at  NRMC  Oakland,  and  about  the  interaction  and 
communication  that  takes  place  between  the  laboratory  and  medical 
staff.  Questions  were  included  with  regard  to  perceptions  about  such 
factors  as: 

•  relations  with  laboratory  personnel; 

•  legibility,  quality,  accuracy  and  format  of 
laboratory  reports; 

•  amount  of  time  required  to  obtain  test  results; 

•  promptness  and  completeness  of  laboratory  reports 
in  patient  records;  and 

•  ease  of  and  amount  of  time  required  to  obtain 
information  by  telephone. 
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Respondents  were  asked  to  rate  satisfaction  with  each  factor.  These 
ratings,  assigned  scale  values  of  5,  4,  3,  2,  1  (the  conventional 

Likert  scale)  were: 

5  highly  satisfactory; 

4  satisfactory; 

3  neither  satisfactory  nor  unsatisfactory; 

2  unsatisfactory;  or 

1  highly  unsatisfactory. 

In  addition,  a  number  of  questions  were  included  inquiring  about 
respondent's  position,  medical  specialty,  and  years  at  NRMC  Oakland. 
A  summary  of  the  responses  by  physicians,  nurses  and  physicians' 
assistants,  and  administrative  staff,  is  presented  in  Table  17. 

b.  Medical  Staff 

A  total  of  183  questionnaires  were  returned  (66.2  percent 
response  rate) .  Of  those  physicians  responding,  the  internal  medicine 
specialty  composed  the  largest  group  (13  percent) .  The  second  largest 
group  was  surgery  (12  percent),  followed  by  pediatrics  (10  percent). 
Approximately  38  percent  of  physicians  had  been  at  the  facility 
between  one  and  three  years  and  24  percent  were  assigned  for  less  than 
one  year.  All  responding  physicians  were  military  officers. 

In  terms  of  usage,  12  percent  felt  that  their  use  in  requesting 
tests  was  heavy.  The  majority  (48  percent)  felt  that  their  use  was 
average,  and  39  percent  felt  that  it  was  light.  Sixty-three  percent 
of  the  respondents  indicated  that  within  the  last  month  they  had 
patient  laboratory  tests  lost.  This  happened  two  or  fewer  times  per 
month  according  to  the  majority  (66  percent)  of  physicians. 

In  the  specific  question  areas  addressed,  physicians  indicated 
concern  for  the  following: 

•  the  promptness  with  which  laboratory  reports  are  filed 
in  patient  charts  (mean  weighted  scale  rating  of  2.57); 

•  the  completeness  with  which  laboratory  reports  are  filed 
in  patient  charts  (2.73); 

•  the  frequency  with  which  laboratory  results  are  lost  (2.71); 

•  the  amount  of  time  required,  once  they  reach  the  laboratory 
by  telephone,  to  obtain  information  about  a  specific  patient 
or  result  (2 . 62) . 
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TABLE  17 


Weighted  mean  response  was  obtained  by  assigning  values  of  1  through  5 
to  the  categories  of  "not  at  all  satisfied"  through  "very  satisfied," 
respectively,  and  dividing  the  sum  by  the  number  of  responses. 

^No  Data  Available  (NDA) . 
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On  the  other  hand,  physicians  tended  to  be  satisfied  with  the 
following: 

•  legibility  of  laboratory  reports  (3.92); 

•  quality  of  information  contained  in  laboratory  reports  (3.88); 

•  format  in  which  information  is  presented  in  laboratory 
reports  (3.73) 

•  accuracy  of  the  information  received  from  the  laboratory 
(3.74); 

•  contribution  of  the  clinical  laboratory  to  patient  care 
(3.91); 

m  confidence  in  the  information  received  from  the  laboratory 
(3.85). 

c.  Nursing  Staff 

The  nursing  questionnaire  was  similar  in  format  to  the  physician 
questionnaire.  A  total  of  146  questionnaires  were  returned, 
representing  a  response  rate  of  60.1  percent.  Approximately  80 
percent  were  from  RNs  and  17  percent  from  LPNs.  The  sample  consisted 
of  67  percent  military  officers  and  31  percent  civilian  nurses.  The 
majority  considered  themselves  average  (49  percent)  or  above  average 
(33  percent)  users  of  the  laboratory  services.  Fourteen  percent 
considered  themselves  below  average  users.  Similar  to  the  physician 
responses,  most  (59  percent)  nurses  felt  that  laboratory  tests  had 
been  lost  in  the  past  month,  although  most  (60  percent)  indicated  this 
happened  two  or  fewer  times  per  month. 

Interestingly,  no  major  differences  were  noted  in  response 
ratings  between  nurses  and  physicians.  Slight  differences  noted 
include: 

•  Nurses  felt  that  the  "promptness  with  which  laboratory 
reports  are  filed  in  patient  charts"  was  somewhat 
satisfactory  (average  scale  rating  3.24),  while  physicians 
felt  it  was  somewhat  unsatisfactory  (2.57). 

•  Similarly,  for  "the  completeness  with  which  laboratory 
reports  are  filed  in  patient  charts"  was  somewhat 
satisfactory  (average  scale  rating  3.24),  while 
physicians  felt  it  was  somewhat  unsatisfactory  (2.57). 
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•  Also,  physicians  were  somewhat  more  satisfied  with  the 
confidence  in  the  information  received  from  the  laboratory 
(3.85)  than  nurses  (3.37). 

d.  Laboratory  Staff 

A  separate  survey  questionnaire  was  administered  to  73  civilian 
and  enlisted  personnel  on  the  laboratory  staff.  The  response  rate  was 
86.3  percent.  Ten  out  of  the  30  questions  in  the  baseline  laboratory 
personnel  questionnaire  which  might  be  affected  by  an  automated 
laboratory  system  were  analyzed: 

•  I  have  enough  time  to  get  the  job  done  (1); 

•  I  have  enough  space  to  work  effectively  (2); 

•  The  work  is  interesting  (6); 

•  I  have  enough  information  to  get  the  job  done  (12); 

•  I  have  enough  equipment  to  get  the  job  done  (13); 

•  My  supervisor  is  competent  in  doing  his/her  job  (15); 

•  My  responsibilities  are  clearly  defined  (17); 

•  The  pay  is  good  (25); 

•  The  chances  for  promotion  are  good  (25); 

•  On  the  whole,  I  am  quite  satisfied  with  my  job  (30). 

The  remaining  questions  dealt  with  the  overall  satisfaction  with 
issues  such  as  ability  to  make  friends,  time  to  get  to  work, 
etc. --issues  judged  to  be  less  affected  by  TRILAB.  For  questions  1, 
12,  13,  17,  the  respondents  "tend  to  agree"  and  "definitely  agree" 
that  they  had  enough  time,  information,  and  equipment  to  get  their 
work  done,  and  that  their  responsibilities  were  clearly  defined  (mean 
composite  scores  of  3.61,  4.05,  3.89,  3.71,  respectively). 

Interestingly,  19  percent  believed  that  they  did  not  have  the  time 
necessary  to  do  their  job.  A  majority  of  the  respondents  (60  percent) 
"tend  to  agree"  that  they  "are  quite  satisfied"  with  their  job; 
however,  40  percent  of  the  respondents  were  either  uncertain  or 
dissatisfied  witv  their  job. 

Although  only  60  percent  were  satisfied  with  their  job,  88 
percent  agreed  that  the  work  was  interesting  (question  6) .  Questions 
25  and  29  attempted  to  examine  the  issues  of  pay  and  promotions.  The 
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data  showed  that  43  percent  definitely  agreed  the  pay  was  low.  A 
total  of  70  percent  either  agreed  that  pay  was  low  or  were  uncertain. 
Similarly,  75  percent  either  disagreed  with  or  felt  uncertain  about 
the  chances  of  promotion. 

2.  Post-Implementation  Stud 


Survey  questionnaires  were  also  distributed  to  all  medical  staff, 
nurses,  administrative  corpsmen  and  clerical  staff,  and  laboratory 
staff  in  the  hospital  during  the  post-implementation  study.  The 
purpose  of  the  questionnaires  was  to  determine  the  degree  of 
satisfaction  of  various  provider  categories  and  staff  with  regard  to 
TRILAB  operations  and  to  compare  their  perceptions  of  operations  of 
the  clinical  laboratory  with  TRILAB,  compared  with  manual  (baseline) 
operations  prior  to  implementation  of  the  TRILAB  system. 

The  questionnaire  included  some  20  questions  regarding  the 
respondents'  perception  of  various  specific  aspects  of  the  TRILAB 
system.  Respondents  were  asked  to  rate  satisfaction  with  each  factor. 
These  ratings,  assigned  scale  values  of  5,  4,  3,  2,  1  (the  Likert 
scale)  were:  5 — highly  satisfactory;  4 — satisfactory;  3 — neither 
satisfactory  nor  unsatisfactory,  or  undecided  no/opinion; 
2 — unsatisfactory;  or  I — highly  unsatisfactory. 

In  addition,  a  number  of  questions  were  included  inquiring  about 
respondents'  overall  satisfaction  with  clinical  laboratory  services 
and  with  the  TRILAB  system;  frequency  of  occurrence  of  problems;  and 
frequency  of  problems  compared  to  previous  manual  laboratory 
operations  (to  be  answered  by  those  staff  who  were  also  at  the 
facility  before  TRILAB  was  installed). 

The  number  of  questionnaires  distributed  and  the  responses  are 
summarized  below: 


Number  Returned 
Number  Sent  and  Utilized 


All  medical  staff 
(MDs  and  dentists) 

Nursing  staff,  including 
physicians'  assistants 

Administrative,  corpsmen 

Laboratory  staff 

Overall 


Response  Rate 


35.3% 


39.1% 
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The  overall  39  percent  response  rate  of  usable  (completed) 
questionnaires  was  less  than  that  achieved  in  the  baseline  period  (64 
percent),  primarily  because  many  potential  respondents  (such  as 
dentists,  psychiatrists,  pharmacists,  etc.)  had  no  familiarity  with  or 
did  not  use  the  TRILAB  system;  a  number  of  uncompleted  questionnaires 
were  returned  with  such  comments. 

Of  the  medical  staff  responding  to  the  questionnaire,  43  percent 
described  themselves  as  "light"  users  of  the  clinical  laboratory,  25 
percent  "moderate"  users  and  11  percent  "heavy"  users.  Of  the  nursing 
staff  responding,  18  percent  described  themselves  as  "light"  users,  36 
percent  "moderate"  users  and  45  percent  "heavy"  users.  Eighty-three 
percent  of  the  administrative  staff  indicated  that  they  were  "light" 
users,  17  percent  "moderate"  users  and  none  indicated  that  they  were 
"heavy"  users. 

b.  Questionnaire  Results:  Users 

Satisfaction  Levels.  A  copy  of  the  questionnaire  sent  to 
physicians,  nurses  and  physicians'  assistants  and  other  administrative 
users  (corpsmen  and  clerks) ,  and  a  detailed  analysis  of  selected 
responses  are  presented  in  Appendix  A.  A  summary  of  the  results  is 
presented  in  this  section. 

Table  18  summarizes  the  weighted  mean  satisfaction  levels  with 
regard  to  various  aspects  of  the  TRILAB  system  for  physicians,  nurses 
and  physicians'  assistants,  and  other  users.  In  general,  users  tended 
to  be  satisfied  with  aspects  of  the  TRILAB  system  such  as  system 
availability,  system  reliability,  and  system  accuracy.  Users  were 
also  satisfied  with  regard  to  clarity,  completeness  and  abnormals' 
identification  in  the  reports  (average  scale  reading  of  approximately 
4.2)  and  somewhat  less  satisfied  with  conciseness  of  the  report  format 
(4.0). 


Users  were  moderately  satisfied  with  system  efficiency,  as 
measured  by  result  turnaround  times  and  training  (3. 0-3. 5),  and  were 
quite  satisfied  with  results  accuracy  (4.0).  Users  were  also 
reasonably  satisfied  with  various  aspects  of  information  retrieval, 
such  as  access  to  laboratory  results,  retrieval  of  previous  data,  ease 
and  timeliness  of  results  retrieval  (3.5-4. 1).  Access  to  STAT 
paper-copy  results  received  a  somewhat  lower  rating  (3.1). 
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TABLE  18 


WEIGHTED  MEAN  SATISFACTION  LEVELS  OF  USERS 
REGARDING  TRILAB  SYSTEM  AT 
NRMC  OAKLAND 


System  Aspect 


TRILAB  System 


System  Availability 
System  Reliability 
System  Accuracy 

Test  Report  Form 


Clarity  of  Printout 
Completeness  of  Results 
Abnormals  Indication 
Conciseness 

System  Efficiency 


STAT  Inpatient 
Turnaround  Time 
Routine  Inpatient 
Turnaround  Time 
STAT  ER  Turnaround  Time 
Routine  Outpatient 
Turnaround  Time 
Special  Case  Flexibility 
Training 

Results  Accurac 


_ Mean  Satisfaction  Level _ 

Nurses  and  Other  Users 

Physicians  Physician  Assistants  (administrative) 


Information  Storage 


and  Retrieval 


Access  to  Laboratory  Results 
Access  to  STAT  Reports 
Retrieval  of  Previous  Data 
Cumulative  Results 
Searching  Patient  Data  Base 
Ease  and  Timeliness 

Overall  Satisfaction 


With  Laboratory  Services 
With  TRILAB 


Average  of  Likert  scale  values 
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Overall,  users  provided  an  average  satisfaction  rating  of  3.6 
with  laboratory  services  and  3.8  with  the  TRILAB  system.  No 
consistent  or  major  differences  were  noted  in  response  ratings  among 
physicians,  nurses  and  other  users. 

Frequency  of  Problem  Occurrences.  Table  19  summarizes  the  users' 
perceived  frequency  of  problems  with  laboratory  services  in  the 
post-implementation  period.  Survey  responses  were  calculated  in  terms 
of  occurring  often,  occasionally,  rarely  or  no  opinion.  (Information 
similar  to  this  was  not  collected  in  the  baseline  survey.)  Excluding 
"not  applicable"  and  "no  opinion"  responses,  the  median  response  was 
"occasionally"  for: 

•  tests  repeated  due  to  delay  in  receiving  original  results; 

•  tests  repeated  due  to  lost  results; 

•  tests  repeated  due  to  inaccurate  results; 

•  frequency  of  phone  calls  to  obtain  results;  and 

•  duplication  of  report  data. 

A  number  of  comments  with  regard  to  report  format  were  made  in 
the  comments  section  (see  below). 

Comparison  of  TRILAB  with  Manual  Operations.  Users  were  asked  to 
compare  relative  frequency  of  problem  events  under  TRILAB  operations 
with  previous  laboratory  (manual)  operations.  Only  those  users  who 
were  at  the  facility  before  TRILAB  was  installed  (in  February  1982) 
were  asked  to  answer  this  question.  Two-thirds  of  the  respondents  to 
the  post-implementation  survey  fell  into  this  category. 

The  TRILAB  system  received  relatively  "high  marks"  for  most 
problem  categories  (Table  20) .  The  median  response  was  that  the 
following  occurred  "less  frequently"  with  the  TRILAB  system: 

•  tests  repeated  due  to  delays  (43.4  percent  felt  they 
occurred  less  frequently); 

•  tests  repeated  due  to  lost  results  (46.7  percent);  and 

•  telephone  calls  to  the  laboratory  (64.6  percent). 

The  median  response  to  "tests  repeated  due  to  inaccurate  results" 
was  that  this  occurred  with  similar  frequency  (40.3  percent). 
Unnecessary  duplication  of  report  data  also  received  a  median  response 
of  "similar  frequency  with  TRILAB;"  as  mentioned  above,  a  number  of 
written  comments  were  made  on  report  format. 
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TABLE  19 


FREQUENCY  OF  EVENTS  RELATING  TO  LABORATORY  TEST  RESULT 
AVAILABILITY  AS  REPORTED  IN  POST- IMPLEMENTATION  SURVEY  OF  USERS 

AT  NRMC  OAKLAND 


Frequency  (Percent  of  Respondents) 

Event  Often  Occasionally  Rarely  No  Opinion 


Tests  repeated  due  to 

delays 

5.6% 

36.4% 

27.9% 

21.5% 

Tests  repeated  due  to 

lost  results 

3.9 

41.9 

28.4 

16.2 

Tests  repeated  due  to 

inaccurate 

results 

5.2 

38.6 

29.6 

17.6 

Telephone  calls  to  laboratory 

18.  5 

46.4 

21.0 

9.4 

Duplication  of  report 

data 

35.1 

20.3 

15.6 

20.8 
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COMPARISON  OF  TRIBAB  WITH  MANUAL  OPERATIONS 
POST-IMPLEMENTATION  PERIOD  OF  USERS 
NRMC  OAKLAND* 


IS 

r(«. 

Change 

in  Frequency 

(Percent  of 

Respondents) 

More  Frequently 
With  TRILAB 

Similar 

Frequency 

Less 

Frequently 

Never 

with  TRILAB 

No 

Opinioi 

* 

Tests  repeated  due 
to  delays 

9.3% 

23.1% 

43.4% 

1.6% 

22.5% 

Tests  repeated  due 
to  lost  results 

6.0 

24.7 

46.7 

2.7 

19.8 

£ 

Tests  repeated  due 

to  inaccurate  results 

0.6 

40.3 

29.3 

2.2 

27.6 

*\ 

* 

Telephone  calls  to 
Laboratory 

6.1 

14.4 

64.6 

2.2 

12.7 

i 

Unnecessary  duplication 
of  report  data 

35.7 

13.2 

24.2 

2.2 

24.7 

★Completed  by  users  who  were  also  at  the  hospital  prior  to  installation  of  TRILAB. 
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Utilization  of  TRILAB  Inquiry  Capabilities.  Users  were  asked  to 
indicate  relative  frequency  of  utilization  of  the  inquiry  capability 
of  the  TRILAB  system  (Table  21).  Physicians  indicated  that  they  used 
the  system  "often"  to  obtain  results  performed  the  previous  day  and 
within  two  to  six  days  and  "occasionally"  for  results  performed  within 
one  or  more  weeks.  Nursing  staff  and  other  users  indicated  that  they 
"occasionally"  utilized  the  system  to  retrieve  results  carried  out  the 
previous  day  and  within  two  to  six  days,  and  "rarely"  for  results 
performed  over  a  week  ago. 

Acceptable  Turnaround  Times.  Users  were  asked  to  indicate 
acceptable  turnaround  times  for  categories  of  tests  results  (Table  22 
and  Figure  3).  Physicians  indicated  on  the  average  they  desired  that 
STAT  inpatient  test  results  be  received  within  0.75  hours  and  that 
STAT  requests  from  the  Emergency  Room  be  received  within  0.54  hours. 
Acceptable  average  turnaround  times  for  routine  inpatient  tests  were 
14.3  hours  and  for  routine  outpatient  tests  30.3  hours.  Nurses  and 
other  users  had  somewhat  lower  acceptable  turnaround  times. 

Comments.  In  addition  to  answering  the  specific  questions, 
respondents  were  asked  to  provide  any  additional  comments  on  the 
TRILAB  system.  Approximately  60  physicians,  or  two-thirds  of 
physician  respondents,  provided  such  additional  comments.  Most  of  the 
comments  referred  to  ways  in  which  the  effectiveness  or  usefulness  of 
the  system  could  be  improved,  from  the  physicians'  point  of  view.  The 
most  frequent  comments,  in  descending  order  of  frequency,  were: 

•  Need  for  More  Terminals.  The  most  frequent  comment  made 
(21  respondents)  was  for  the  need  for  more  terminals, 
particularly  in  areas  such  as  Urology,  and  the  Dental,  ENT 
and  Pediatric  Clinics,  which  did  not  have  TRILAB  terminals. 

•  Format  of  Computer  Printout  The  next  most  frequent  comment 
(14  respondents)  was  that  the  computer  printouts  were  too 
large  and  redundant.  For  example,  it  was  felt  that  it  was 
not  useful  to  include  pending  results  on  the  reports,  and 
that  a  whole  sheet  of  paper  was  not  necessary  to  provide  a 
single  result. 
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TABLE  21 

FREQUENCY  OF  TRILAB  INQUIRY  TO  OBTAIN  TEST  RESULTS 
POST- IMPLEMENT I ON  SURVEY  AT  NRMC  OAKLAND 


Time  Period  When 
Patient  Test  Was 
Performed 

Previous  Day 

Previous  2-6  Days 

Previous  1-2  Weeks 

Over  2  Weeks  Previously 


Median  Response 


Nurses  (Other  Users 

Physicians  Physician  Assistants  (administrators) 


Often 

Often 

Occasionally 

Occasionally 


Occasionally 

Occasionally 

Rarely 

Rarely 


Occasionally 

Occasionally 

Rarely 

Rarely 


TABLE  22 

ACCEPTABLE  TURNAROUND  TIMES 
POST-IMPLEMENTATION  PERIOD  SURVEY  AT  NRMC  OAKLAND 


$ 

■ 

Acceptable  Mean  Turnaround 

Time  (hours) 

Test  Type 

Physicians 

Nurses, 

Physician  Assistants 

Other  Users 
(administrative) 

IN 

k 

STAT  Inpatient 

0.  75 

0.57 

0.57 

Routine  Inpatient 

14,30 

11.  20 

10.90 

3 

STAT  ER 

0.54 

0.35 

0.  30 

5 

Routine  Outpatient 

30.30 

20.90 

30.00 
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Percent  Percent  Percent  Percent 


Hours 


FIGURE  3  ACCEPTABLE  TEST  TURNAROUND  TIMES 

POST-IMPLEMENTATION  SURVEY  AT  NRMC  OAKLAND 
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•  Need  for  Nuclear  Medicine  Results.  Six  respondents  felt 
that  the  addition  of  the  Nuclear  Medicine  laboratory 
results  on  the  system  was  very  important. 

•  Patient  Address/Telephone  Number.  It  was  felt  by  six 
respondents  (clinic  physicians)  that  it  would  be  very  useful 
to  have  patients'  telephone  numbers  and  addresses  on  the 
printout,  so  that  the  patient  could  be  notified  more  easily, 
e.g.,  if  results  were  abnormal. 

•  Delay  in  Getting  Hard-Copy.  Six  respondents  felt  that  STAT 
turnaround  time  for  the  hard-copy  results  was  too  long. 

•  STAT  Turnaround  Time  Too  Long.  Three  respondents  felt  that 
STAT  results  turnaround  time  was  too  long. 

•  Formal  Training.  Four  respondents  expressed  the  need  for 
a  better  training  program  on  utilization  of  the  terminals 
and  for  an  instruction  manual. 

•  System  Nonfunctioning.  Three  physician  respondents  commented 
on  the  fact  that  the  system  was  down  too  frequently. 

•  Inadequate  Memory.  Three  respondents  commented  on  the 
insufficient  archival  capacity  of  the  system  for  results. 

Approximately  50,  or  60  percent,  of  the  nursing  staff 
questionnaires  had  comments  with  respect  to  the  TRILAB  system 
operations.  Several  of  the  comments  indicated  that  the  system  had 
improved  service  from  the  laboratory,  citing  the  fact  that  there  were 
fewer  telephone  calls  required  to  obtain  results  and  a  resultant 
reduction  in  staff  workload.  However,  most  of  the  comments  dealt  with 
areas  in  which  service  could  be  improved: 

•  Multiple  Terminal  Beeps.  Twelve  of  the  questionnaires 
mentioned  the  fact  that  their  terminal  "beeped"  unneces¬ 
sarily,  taking  staff  away  from  their  regular  duties.  It 
was  felt  by  three  respondents  that  beeping  for  incomplete 
results  was  inappropriate;  they  needed  complete  laboratory 
results  before  being  able  to  take  any  action.  The  system 
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too  frequently  beeped  for  STAT  results  which  had  already 
been  received,  or  when  results  were  being  "dumped"  (cleared 
from  the  system);  that  is,  there  appeared  to  be  hardware  or 
software  problems  which  resulted  in  unnecessary  beeps. 

•  More  Terminals.  The  second  most  frequent  comment  (nine 
respondents)  expressed  the  need  for  terminals  in  areas 
(such  as  the  operating  room),  which  did  not  have  terminals. 

•  STAT/Urgent  Turnaround  Times.  Five  respondents  commented 
that  the  turnaround  time  for  STAT/urgent  requests  was 
frequently  too  long. 

•  Matching  of  Results  with  Patient  Records.  Five  staff 
commented  on  the  fact  that  laboratory  results  often 
were  filed  under  an  incorrect  patient  record,  e.g.,  the 
incorrect  admission  for  a  patient  who  had  had  multiple 
admissions  to  the  facility.  This  increased  the  tine 
required  to  find  the  result. 

•  Training.  Four  respondents  commented  on  the  lack  of 
adequate  training  received  by  ward  staff,  and  an 
additional  two  commented  on  the  fact  that  there  were 

no  operating  instructions  provided  for  use  of  the  terminals. 

•  Interim  Report  Turnaround  Time.  Three  respondents 
commented  that  frequently  the  interim  reports  for 
pre-operative  patients  were  not  available  on  time. 

Twenty-three  of  the  questionnaires  by  corpsmen,  clerks  and 
hospital  technicians,  or  about  40  percent  of  the  questionnaires  in 
this  category,  had  comments  with  regard  to  the  operation  of  the  TRILAB 
system: 

•  Report  Redundancy.  Five  respondents  felt  that  the  reports 
produced  by  the  system  were  frequently  superfluous  or 
redundant,  reporting  the  same  test  result. 

•  Multiple  Beeps.  Four  of  the  respondents  commented  on  the 
multiple  beep  problem  of  the  terminals. 
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•  Training.  Three  corpsmen  pointed  out  that  the  corpsmen  had 
received  either  insufficient  or  no  training  in  utilization 
of  the  system. 

The  comments  received  by  and  large  mirrored  those  of  the  nursing 
staff. 

c.  Questionnaire  Results:  Laboratory  Staff 

A  separate  survey  questionnaire  was  administered  to  the 
personnel  on  the  laboratory  staff.  A  copy  of  the  questionnaire  and 
detailed  analysis  of  survey  results  are  presented  in  Appendix  A. 

Satisfaction  Ratings.  Laboratory  staff  were  asked  to  indicate 
their  satisfaction  rating  with  regard  to  a  number  of  characteristics 
of  the  TRILAB  system.  The  results  are  summarized  in  Table  23.  On 
average,  laboratory  staff  did  not  assign  as  high  a  satisfaction  rating 
to  the  TRILAB  system  as  did  users.  In  particular,  only  average 
satisfaction  was  expressed  with  regard  to  system  availability, 
reliability,  ease  of  order  entry,  results  entry,  and  logging  functions 
(scale  ratings  of  2. 5-2. 9).  Somewhat  higher  satisfaction  levels  were 
expressed  with  regard  to  system  accuracy,  clarity  of  printout,  inquiry 
capability,  file  edit  capability,  and  abnormals  identification 
(ratings  of  3. 1-4.1). 

Training  and  perceptions  of  medical  staff  familiarity  with  the 
system  received  lower  scale  ratings  (2.8).  Screen  legibility  was 
considered  very  satisfactory  (4.5),  but  command  functions  and  terminal 
locations  received  lower  ratings  (3.3  and  2.7). 

Moderate  satisfaction  was  expressed  with  regard  to  information 
storage  and  retrieval,  quality  control  data  and  quality  control 
reporting  (2. 8-3. 4). 

Overall  satisfaction  with  the  TRILAB  system  was  neutral  (2.4), 
but  above  average  satisfaction  was  reported  for  overall  laboratory 
services  (3.6). 

Frequency  of  Problems.  With  regard  to  frequency  of  problem 
occurrences  (Table  24),  most  respondents  indicated  that: 

•  repeating  of  tests  due  to  lost  results  occurred  only 
occasionally  (42  percent)  or  rarely  (42  percent); 
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TABLE  23 

WEIGHTED  MEM  SATISFACTION  LEVELS  BY  LABORATORY  PERSONNEL 
REGARDING  TRILAB  SYSTEM 
NRMC  OAKLAND 


System  Aspect 

A.  TRILAB  System 

System  Availability 
System  Reliability 
System  Accuracy 

B.  Test  Accessioning 

Ease  of  Test  Order  Entry 
Label  Generation 
Worksheet  Clarity 

C.  Results  Report 

Print-out  Clarity 
Results  Entry 
Inquiry  Capability 
Hard-Copy  Results  Availability 
File  Edit  Capability 

D.  Efficiency 

Abnormals  Identification 
Department  Log 
Medical  Staff  Familiarity 
Training 

E.  CRT  Functions 

Screen  Legibility 
Command  Functions 
Terminal  Locations 

F.  Results 

Data  Quality  Control 
Results  Accuracy 
Quality  Control  Reporting 

G.  Information  Storage  and  Retrieval 

Verifying  Patient  ID 
Results  Retrieval 
Results 

Cumulative  Results 

Overall  Satisfaction 

With  TRILAB 
Laboratory  Services 


Weighted  Mean„ 
Scale  Rating” 


^sing  the  Likert  scale. 
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TABLE  24 


FREQUENCY  OF  PROBLEM  OCCURRENCES 

POST-IMPLEMENTATION  PERIOD  SURVEY  OF  LABORATORY  PERSONNEL  AT  NRMC  OAKLAND 


Often 

Occasionally 

Rarely 

Never 

No  Opinion 

Tests  repeated  due  to  lost 
results 

0.1% 

42.1% 

42.1% 

13.2% 

2.6% 

Duplication  of  information 

21.1 

28.9 

36.8 

2.6 

10.5 

Telephone  calls  to  units 

5.3 

73.7 

15.8 

2.6 

2.6 

Time  spent  on  manual  record 
keeping 

44.  7 

28.9 

15.8 

2.6 

7.9 

Time  spent  telephoning  STAT 
results  to  unit 

13.2 

40.0 

28.9 

2.6 

5.3 

Errors  in  transcription 

15.8 

44.  7 

23.7 

5.3 

10.5 
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•  duplication  of  information  occurred  rately  (37  percent); 

•  telephone  calls  to  units  occurred  occasionally  (74  percent); 

•  errors  in  transcription  also  only  occurred  occasionally 
(45  percent) ;  and 

•  time  spent  on  manual  record-keeping,  however,  occurred 
"often"  (45  percent). 

When  asked  to  compare  operations  under  TRILAB  with  manual 
operations  prior  to  installation  of  TRILAB  (Table  25) ,  respondents 
(who  had  been  with  the  laboratory  prior  to  installation  of  TRILAB) 
indicated  that  the  following  occurred  less  frequently  with  TRILAB: 

•  telephone  calls  to  providers  (73  percent);  and 

•  time  spent  on  manual  record-keeping  (47  percent). 

The  following  were  judged  to  occur  with  similar  frequency  under 
TRILAB: 

•  repeating  tests  due  to  inaccurate  results  (53  percent) ;  and 

•  transcription  discrepancies  (43  percent) . 

TRILAB  Imp  rovemen  t  s .  Respondents  were  asked  to  identify  the 
relative  importance  of  improvements  due  to  TRILAB  (Table  26) . 
Respondents  felt  that  improvement  in  format  of  laboratory 
requests/results  was  either  somewhat  important  (30  percent)  or 
somewhat  unimportant  (30  percent). 

Improvement  in  efficiency  of  laboratory  operations  was  considered 
somewhat  important  (34  percent) .  Most  respondents  were  undecided 
about  accuracy  of  results  under  TRILAB  (36  percent),  but  felt  that 
improvement  in  ease  of  information  storage  and  retrieval  was  very 
important  (46  percent) ;  respondents  assigned  a  higher  value  to  this 
aspect  in  this  question  than  implied  in  a  previous  question  (Table  23, 
item  G) .  Improvement  in  number  of  telephone  calls  was  judged  either 
somewhat  important  (40  percent)  or  very  important  (30  percent). 

Instrumentation.  In  response  to  the  question  "Are  there  any 
instruments  which  should  be  on-line  to  the  TRILAB  system  but  are  not 
now?",  the  two  instruments  mentioned  most  often  were  the  Coagamate 
(suggested  by  seven  respondents)  and  the  Differential  Counter 
(mentioned  by  five  respondents).  The  Gemini  (four  respondents)  was 
also  suggested.  (Interfaces  were  available  for  the  Coagamate  and 
Gemini,  but  were  not  being  utilized.) 
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COMPARISON  OF  TRILAB  WITH  MANUAL  OPERATIONS 
POST-IMPLEMENTATION  PERIOD  OF  LABORATORY  PERSONNEL 

NRMC  OAKLAND 

_ Change  In  Frequency  (Percent  of  Respondents) _ 

More  Frequently  Similar  Less  Never  No 

With  TRILAB  Frequency  Frequently  with  TRILAB  Opinion 


Telephone  calls  to 
inpatient  units/ 


* 

outpatient  clinics 

6.7% 

20.0% 

73.3% 

0% 

0 

§ 

Duplication  of 
information 

20.7 

17.2 

41.4 

10.3 

10.3 

Necessity  of  repeating 
tests  due  to 
inaccurate  results 

3.3 

53.3 

30.0 

13.3 

10.0 

i 

Time  spent  on  manual 
record  keeping 

20.0 

33.3 

46.7 

0 

0 

Discrepancies  in 
transcription 

23.3 

43.3 

20.0 

3.3 

10.0 

TABLE  26 

IMPROVEMENTS  DUE  TO  TRILAB 

POST- IMPLEMENTATION  PERIOD  SURVEY  OF  LABORATORY  PERSONNEL  AT  NRMC  OAKLAND 


Percent  of  Respondents  Expressing  Opinion 
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Very 

Somewhat 

Somewhat 

Very 

Undecided/ 

Important 

Important 

Unimportant 

Unimportant 

No  Opinion 

Format  of  lab  requests /results 

10.8 

29.7 

29.  7 

10.8 

18.9 

Si 

Efficiency  of  laboratory 

operations 

14.  3 

34.  3 

5.7 

17.1 

28.6 

*  .% 

Accuracy  of  results 

25.0 

13.9 

5.6 

19.4 

36.1 

Ease  of  information  storage 

and  retrieval 

45.9 

37.8 

5.4 

2.7 

8. 1 

Number  of  telephone  calls 

29.  7 

40.5 

5.4 

2.  7 

21.6 
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Two  respondents  indicated  that  the  C-800  should  not  be  on-line, 
and  one  respondent  indicated  that  the  SMAC  should  not  be  on-line. 

Additional  Comments.  The  most  frequent  additional  comments 
were  the  following: 

•  System  Too  Slow.  The  comment  made  most  often  (by  16 
respondents)  was  that  the  system  was  too  slow  for 
accessioning  and  entering  results,  particularly  at 
certain  peak  times  of  the  day.  Respondents  specifically 
mentioned  the  fact  that  increased  memory  was  required. 

•  Down  Time.  Six  respondents  made  the  comment  that  the 
computer  system  broke  down  too  often,  increasing  the 
workload  due  to  resultant  recording  of  results. 

•  Preventive  Maintenance.  Four  respondents  objected  to 
preventive  maintenance  being  performed  during  the  day, 
when  utilization  was  the  highest,  instead  of  on  evenings 
or  weekends. 

On  the  positive  side,  advantages  cited  were  easier  access  to 
results  (three  respondents)  and  fewer  telephone  calls  (two 
respondents).  Five  respondents,  however,  felt  that  the  TRILAB 
computer  system  resulted  in  extra  work  for  the  laboratory. 

d.  Comparison  of  Satisfaction  Ratings  in  Baseline  and 

Post-Implementation  Periods 

Users.  Table  27  compares  the  average  satisfaction  ratings  with 
laboratory  services  between  the  two  study  period  surveys.  Average 
scale  ratings  by  physicians  increased  by  0.4  to  0.6  for  the  following 
aspects  of  laboratory  services: 

•  accuracy  of  results; 

•  legibility; 

•  length  of  time  between  routine  tests  and  results;  and 

•  length  of  time  between  STAT  tests  and  availability  of  results. 

Completeness  of  results  and  overall  laboratory  performance 

received  lower  rating  increases  (0.3  and  0.1,  respectively). 
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Nursing  staff  showed  a  higher  increase  in  average  satisfaction 
ratings  (0.7  or  0.8)  for  accuracy,  legibility,  completeness,  and 
length  of  time  between  routine  tests  and  results,  and  smaller 
increases  in  satisfaction  ratings  (0.1-0. 2)  for  length  of  time  between 
STAT  tests  and  results,  and  overall  laboratory  performance. 

Administrative  users  showed  a  large  increase  (1.0)  in 
satisfaction  rating  for  completeness  of  results,  and  a  smaller 
increase  in  satisfaction  with  overall  laboratory  performance  (0.1)  and 
legibility  (0.4). 

These  results  indicate  that  user  satisfaction  with  aspects  of 
laboratory  performance  had  increased  with  implementation  of  the  TRILAB 
system.  The  degree  of  increase  in  satisfaction  and  particular  area 
emphasized  depended  on  the  user  category. 

Clinical  Laboratory  Personnel.  The  baseline  questionnaire 
included  only  questions  related  to  general  aspects  of  job  satisfaction 
that  are  unrelated  to  implementation  of  the  TRILAB  system. 
Consequently,  the  results  reported  in  Section  c  for  the  laboratory 
staff  post-implementation  questionnaire  are  believed  to  present  the 
best  assessment  of  laboratory  staff  comparative  satisfaction. 

As  discussed  above,  laboratory  personnel  were  overall  less 
satisfied  with  the  TRIMIS  system  than  were  providers,  indicating  that 
the  system  was  frequently  slow  and  had  too  much  down  time.  On  the 
positive  side,  however,  laboratory  staff  indicated  that  manual 
record-keeping  and  time  spent  on  phone  calls  had  decreased,  and  that 
results  retrieval  was  easier  with  the  TRILAB  system. 

E.  PATIENT  SATISFACTION  WITH  LABORATORY  SERVICES 
1.  Baseline  Survey 

In  an  attempt  to  measure  satisfaction  of  patients  with  regard  to 
laboratory  services,  the  baseline  study  included  a  questionnaire 
survey  of  outpatients  who  use  the  services  of  the  laboratory.  For  a 
period  of  two  weeks,  25  forms  were  distributed  daily  to  patients  who 
came  to  the  laboratory  for  blood  drawing.  The  first  week  of  sampling 
included  morning  visits,  while  the  second  week  included  afternoon 
visits.  The  volume  of  walk-in  traffic  was  noticeably  higher  in  the 
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morning  and  there  was  more  success  in  having  patients  respond  during 
this  time.  One  hundred  seventy-four  patient  questionnaires  were 
returned,  representing  a  response  rate  of  approximately  70  percent. 

The  patient  questionnaire  included  questions  with  regard  to  the 
patient's  perception  with  regard  to  such  factors  as: 

•  courtesy  of  clinical  laboratory  staff; 

•  amount  of  time  having  to  wait  to  register  with  the 
laboratory  receptionist; 

•  waiting  time; 

•  personal  attention; 

•  amount  of  information  given  about  a  particular  test 
and  the  adequacy  of  the  instructions; 

•  competence  of  the  laboratory  staff  and  test  results;  and 

•  general  satisfaction. 

Patients  were  asked  to  indicate  how  satisfied  they  were  and 
ratings  of  5,  4,  3,  2,  1  were  assigned  (the  Likert  scale).  The 

majority  who  responded  (78  percent)  were  either  retired  or  a  dependent 
of  a  retiree. 

Approximately  86  percent  of  respondents  indicated  that  they  had 
previously  used  the  laboratory  services.  Approximately  five  percent 
indicated  that  they  had  had  to  have  tests  repeated  because  the  results 
had  not  reach  the  physicians. 

On  average,  no  question  had  a  satisfaction  rating  of  less  than 
4.12.  Overall  the  majority  of  patients  were  either  satisfied  or  very 
satisfied  with  the  services  of  the  laboratory. 

2.  Post-Implementation  Survey 

During  the  post-implementation  period  survey,  200  questionnaires 
were  also  distributed  to  patients  who  came  to  the  laboratory  for  blood 
drawing. 

The  patient  questionnaire  included  questions  with  regard  to 
patients'  general  satisfaction  with  clinical  laboratory  services, 
waiting  time  after  registration,  and  waiting  time  for  laboratory 
technicians . 
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Table  28  compares  average  patient  satisfaction  ratings  reported 
in  the  two  surveys.  The  laboratory  received  comparatively  high 
ratings  by  patients  in  both  study  periods.  The  general  satisfaction 
rating  remained  generally  unchanged  (4.5),  as  did  satisfaction  with 
waiting  times  for  specimen  taking  (4.2)  There  was  a  decrease  in 
satisfaction  rating  of  waiting  time  required  to  register  (from  4.5  to 
4.1). 

Patients  were  also  asked  to  indicate  whether  they  had  to  have 
tests  repeated  due  to  lost  results  or  whether  they  experienced  delays 
due  to  incomplete  test  request  forms.  Table  29  summarizes  the 
responses.  Most  patients  indicated  that  they  "never"  or  "rarely" 
experienced  such  problems.  Two  to  three  percent  indicated  that  they 
occurred  "often." 

About  22  percent  of  patients  added  comments  to  their 
questionnaire.  The  majority  of  these  were  laudatory,  and  referred  to 
the  "excellent  service"  received  at  the  facility  or  that  they  had 
experienced  no  problems.  Seven  patients  felt  that  the  waiting  times 
for  service  were  excessive,  or  that  it  took  too  long  to  get  a  clinic 
appointment  (three  patients). 

F.  SYSTEM  COSTS 

The  following  estimates  of  costs  of  the  TRILAB  system  are  based 
on  three  sources:  the  vendor  proposal  for  the  system  at  NRMC  Oakland, 
the  monthly  material  inspection  and  receiving  reports  for  the  lease 
and  maintenance  of  the  systems,  and  interviews  held  with  systems, 
laboratory  and  nursing  staffs  of  the  hospital. 

Costs  of  the  system  are  divided  into  one-time  costs  (including 
system  acquisition,  site  preparation,  set-up  and  training  costs),  and 
recurring  costs  (including  system  operating  staff,  data  processing 
supplies,  system  maintenance  and  space  costs). 

1.  One-Time  Costs 

a.  System  Acquisition 

The  laboratory  system  was  purchased  under  a  13-month  lease- 
to-ownership  plan  at  a  monthly  charge  of  $39,182.  The  total  cost  of 
the  hardware  system,  including  mainframe  and  peripherals  (42  CRTs, 
seven  character  printers  and  one  line  printer)  was  13  months  x 
$39, 182/month  ■  $509,366.  Other  one-time  vendor  charges  were: 
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TABLE  28 


COMPARISON  OF  AVERAGE  PATIENT  SATISFACTION  RATINGS3 
BASELINE  AND  POST- IMPLEMENTATION  PERIOD  SURVEYS  AT  NRMC  OAKLAND 


Baseline 

Post-Implementation 

Difference 

Time  to  register 

4.5 

4.1 

-0.4 

Waiting  time  for  specimen  taking 

4.2 

4.2 

0 

General  satisfaction 

4.6 

4.5 

-0.1 

^Average  of  Likert  scale  ratings. 


TABLE  29 

ESTIMATED  FREQUENCY  OF  PROBLEM  OCCURRENCES 
POST- IMPLEMENTATION  PERIOD  PATIENT  SURVEY  AT  NRMC  OAKLAND 


Don ' t  Know 


Other 

Occasionally 

Rarely 

Never 

NA 

Test  repeated  due  to 
lost  results 

2.0  % 

15. 3% 

23.0% 

53.1% 

6.6%  ' 

Delay  due  to  incomplete 
test  request  form 

3.1 

7.  3 

19.2 

63.2 

7.3 
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System  installation  $  48,895 
Software  188,897 
Documentation  15,790 
Training  21,053 


The  total  system  acquisition  cost  was  therefore  $784,001. 

b.  Site  Preparation 

The  cost  of  preparing  the  computer  room  for  the  TRILAB  and  other 
computer  systems  was  $163,000.  In  addition,  it  was  planned  to  upgrade 
the  air-conditioning  and  power  supply  to  the  computer  room  in  December 
1982;  the  cost  of  site  upgrade  was  estimated  at  $165,000.  Total  cost 
of  site  preparation  was  therefore  $328,000.  Since  the  computer  room 
will  support  several  systems,  30%  of  the  site  preparation  costs  was 
allocated  to  TRILAB.  Site  preparation  cost  was  therefore  estimated  at 
$100,000. 

c.  Other  Equipment 

The  MIS  Department  purchased  additional  anchors  for  the  terminals 
($806)  and  video  tape  ($453)  for  the  system,  for  a  total  of  $1,259. 

d.  Systems  Staff 

During  installation  and  implementation  of  the  system, 

six-man-months  of  analysts  were  involved  (four  man-months  of  an 

*  * 

analyst  GS  11  and  two  man-months  GS  12:  ((4  x  $3,179  )  +  (2  x$3,560  ) 

*  $19,836. 

e.  Training  Time 

The  following  categories  of  hospital  staff  received  training  on 
use  of  the  system,  during  their  normal  duty  hours.  The  estimated 
costs  of  the  staff  time  involved  are  presented  below: 


* 

Civilian  salaries  from  Federal  Employees'  Salary  Scale  (FY  '82)  Step 
4.  Includes  leave  allowance  of  20.9%  and  fringe  of  21.7%.  Military 
salaries  include  base  pay  from  annual  composite  salary  schedule  (FY 
'81)  adjusted  by  the  percentage  increase  reported  for  FY  '82  (Army 
Times ,  October  18,  1981.  Rates  were  adjusted  by  leave  allowance  of 
20.9%  and  38%  fringe  plus  quarters  and  other  special  pay.  Quarters 
and  other  special  pay  were  assumed  to  increase  at  the  same  rate  as 
base  pay. 
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Computer  Operators.  Five  computer  operators  (GS  7  level) 
received  24  hours  of  training  each.  The  staff  time  salary  cost 
therefore  was:  5  x  24  x  $12.39*  =  1,487. 

Laboratory  Staff.  Approximately  45  laboratory  staff  received 
about  five  hours  of  training  each  on  use  of  the  system,  for  a  total  of 
225  hours.  In  addition,  senior  laboratory  staff  spent  approximately 
70  hours  in  training  physicians  to  use  the  terminals  for  inquiry,  on  a 
one-on-one  basis.  The  total  estimated  time  cost  was  295  x  $11.01  = 
$3,248. 

Nursing  Staff.  One  hundred-eleven  members  of  the  Nursing 
Department  staff  received  one  hour  of  formal  training  each.  At  an 
average  hourly  salary  of  $13.11*,  the  salary  time  cost  was  111  x 
$12.68*  =  $1,407. 

Physicians.  As  discussed  above,  approximately  70  physicians 
received  one-on-one  training  from  laboratory  staff.  Assuming  an 
average  0-4  grade  and  one  hour  training,  the  time  cost  involved  was  70 
x  $30.31  =  $2,122. 

Total.  Total  estimated  cost  of  staff  time  involved  in  training 
was  therefore  $8,264. 

File  Building.  Laboratory  staff  time  involved  in  building  the 
initial  data  files  was  estimated  to  total  120  days.  At  an  average 
daily  cost  of  $88.15*,  the  total  staff  time  cost  was  120  x  $88.15  = 
$10,758. 

The  total  one-time  cost  for  installation  and  implementation  of 
the  TRILAB  system  at  NRMC  Oakland,  as  summarized  in  Table  30,  was 
$856,000. 

2.  System  Operating  Costs 

a.  System  Maintenance 

The  initial  monthly  system  maintenance  costs  were: 


Hardware  $2,418 
Software  1,390 
Communication  lines  558 

Total  $4,096 


*See  footnote  page  75. 
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Hardware  maintenance  was  scheduled  to  increase  annually  by  7 
percent,  7  percent,  9  percent,  9  percent,  12  percent,  13  percent,  and 
14  percent  on  July  1,  1982  through  July  1,  1988.  Communication  line 
maintenance  cost  was  scheduled  to  escalate  at  10  percent  per  annum 
commencing  July  1,  1982.  Software  maintenance  cost  was  not  subject  to 
escalation  (a  75  percent  discount  will  be  applied  for  months  50 
through  96  of  the  systems'  life  if  all  "mandatory  and  optional 
systems"  are  purchased  it;  it  is  not  expected  that  this  clause  will  be 
operative).  The  projected  schedule  of  maintenance  cost  is  shown  in 
Table  31. 

b.  Space  Costs 

The  allocated  annual  space  cost  for  the  computer  room  at  NRMC 
Oakland  was  $5.71/sq.  ft.  for  1,560  sq.ft.,  or  $8,908.  There  are 
currently  three  computer  systems;  it  is  planned  that  there  will  be  ten 
systems  within  four  years.  The  space  costs  were,  therefore,  estimated 
at  $2,969  annually  for  the  first  three  years  and  $891  per  year 
thereafter. 

c.  Staff  Costs 

Analyst.  A  full-time  analyst  (GS  11)  was  devoted  to  maintenance 
of  the  system.  The  annual  cost  was  therefore  $38,142. 

Operators.  The  equivalent  of  56  hours  per  week  of  operator  time 
(GS  7)  were  allocated  to  the  system.  The  annual  cost  was  therefore  52 
weeks  x  56  hours  x  $12.39  =  $36,080. 

The  total  annual  system  staff  costs  were  therefore  $74,222. 

d.  Supplies.  Estimated  costs  of  supplies  (primarily  computer 
report  paper)  was  $14,000. 

The  total  estimated  initial  annual  recurring  system  costs  were 
therefore  $140,343,  as  shown  in  Table  32. 
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TABLE  31 


TRILAB  ANNUAL  COMPUTER  MAINTENANCE  COSTS 
NRMC  OAKLAND 


Hardware 

Software 

Line 

Total 

Annual 

Initial 

$2, 148/mo. 

$1,3 90/mo. 

$  558/mo. 

$4, 096/mo. 

$49,152 

July  1,  1982 

2,298 

1,390 

614 

4,302 

51,614 

July  1,  1983 

2,459 

1,390 

675 

4,524 

52,288 

July  1,  1984 

2,681 

1,390 

743 

4,814 

57,768 

July  1,  1985 

2,922 

1,390 

817 

5,129 

61,548 

July  1,  1986 

3,272 

1,390 

899 

5,561 

66,732 

July  1,  1987 

3,698 

1,390 

989 

6,077 

72,924 

July  1,  1988 

4,216 

1,390 

1,087 

6,693 

80,316 

TABLE  32 

ANNUAL  RECURRING  TRILAB  SYSTEM  COSTS 
NRMC  OAKLAND— 1982 


Maintenance 

$  49,152 

1 

Space 

2,969 

Systems'  Staff 

74,222 

v; 

Supplies 

14,000 

Total  $140,343 
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IV.  DISCUSSION  OF  RESULTS 


A.  SYSTEM  BENEFITS 

In  this  section  the  benefits  that  have  accrued  as  a  result  of  the 
TRILAB  system  are  summarized.  First  the  benefits  that  can  be 
quantified  are  presented,  and  then  the  benefits  which  cannot  be 
quantified.  The  benefits  are  based  on: 

•  interviews  with  providers  and  laboratory  staff,  carried 
out  during  implementation  monitoring  visits  and  the 
post-implementation  survey; 

•  data  collected  as  part  of  the  post-implementation  study. 

1.  Inpatient  Units 

Work  sampling  was  not  carried  out  during  the  baseline  evaluation 
in  user  locations.  In  order,  therefore,  to  obtain  estimates  of 
benefits  to  users  (health  care  providers)  of  the  TRILAB  system, 
interviews  were  held  with  nursing  staff  (RNs  and  ward  clerks)  of  six 
inpatient  units,  which  had  a  TRILAB  terminal  for  results  reporting  and 
inquiry. 

Benefits  were  described  as  follows: 

•  Reduced  Time  on  the  Telephone.  This  included  the  time  that 
nursing  staff  spent  on  the  telephone  to  receive  test  results 
and  to  inquire  about  late  or  missing  results.  Nursing  staff 
estimated  that  availability  of  the  terminal  saved  an  average 
of  3-1/2  hours  per  (24-hour)  day  per  unit. 

•  Filing  Time.  It  was  estimated  that  having  cumulative  reports 
saved  on  average  1/2  hour  per  day  per  unit  in  filing,  compared 
to  the  previous  system  of  filing  the  individual  requisition/ 
results  slips. 

•  Chart  Review.  Staff  of  one  unit  estimated  that  availability 
of  cumulative  reports  saved  1/2  hour  per  day  in  chart  review, 
compared  to  having  to  go  through  a  number  of  filed  results 
slips. 

It  was  thus  estimated  that  on  average  4  staff  hours  per  inpatient 
unit  per  day  were  saved,  which  were  made  available  for  patient  care  or 
other  activities. 
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Since  12  inpatient  units  have  terminals,  it  is  estimated  that  48 
hours  per  day  of  nursing  staff  (RN/corpsman/ward  clerk)  time, 
equivalent  to  4  FTEs,  were  saved  per  day  on  inpatient  units  at  NRMC 
Oakland,  due  to  availability  of  the  system. 

•  Duplicate  Tests.  The  nursing  staff  estimated  that  an 
average  of  3.5  duplicate  tests  per  week  per  unit  were 
avoided,  through  being  able  to  determine  the  status  of 
laboratory  test  requests  and  results.  For  the  12  nursing 
units  having  terminals,  it  was  estimated  that  approximately 
42  duplicate  tests  per  week  have  been  avoided  as  a  result 
of  the  system. 

2.  Outpatient  Services 

Nursing  staff  in  the  emergency  room  and  four  outpatient  clinics 
that  have  TRILAB  terminals  were  interviewed  in  order  to  obtain  their 
estimates  of  benefits  achieved  due  to  implementation  of  the  TRILAB 
system. 

Nursing  staff  estimated  that  in  total  an  average  of  4.6  hours  of 
staff  time  per  clinic  per  day  were  saved,  due  to  availability  of 
TRILAB.  The  major  source  of  savings  was  in  reduced  time  required  for 
telephoning  the  laboratory  (2.8  hours  per  day).  The  remaining  savings 
were  due  to  reduced  time  spent  visiting  the  laboratory  to  obtain 
results  or  determine  status  of  tests,  in  filing  and  in  looking  up 
results. 

Since  there  were  six  outpatient  clinics  (including  the  ER)  with 
terminals,  the  estimated  savings  in  staff  time  were  6  x  4.6  =  27.6 
hours  of  staff  time  per  day  (3.5  FTEs). 

3.  Other  User  Benefits 

a.  Nursing  Staff 

Nursing  staff  cited  the  following  benefits,  in  addition  to  those 
described  above: 

•  Improved  Morale.  As  a  result  of  being  able  to  look  up 
test  status  on  the  terminal,  and  the  reduction  in 
telephone  calls  to  the  laboratory,  relationships  between 
nursing  and  laboratory  staff  had  improved  considerably. 

This  had  improved  morale  of  nursing  staff. 
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Decreased  Turnaround  Time.  Turnaround  time  for  test 


results,  especially  for  routine  tests,  had  been  reduced, 
contributing  to  the  reduction  in  telephone  calls  and 
improvement  of  relations  with  the  laboratory.  This  may 
also  have  resulted  in  improved  patient  care. 
Identification  of  Abnormals.  Because  abnormal  results 


were  identified  (by  an  asterisk) ,  leading  to  faster  and 
easier  identification  of  patient  problems,  patient  care 
has  been  improved. 

Discontinuance  of  Flow  Sheets.  Since  the  laboratory  test 


report  formats  are  cumulative,  some  nursing  units  have 
discontinued  the  manual  charting  of  "flow  sheets."  Other 
units,  however,  preferred  to  maintain  them  because  of  their 
more  concise  format  compared  to  the  computer  reports. 

b.  Physicians 

Informal  interviews  were  held  with  a  number  of  physicians. 
Physicians  were  generally  pleased  with  the  system,  once  most  of  the 
"bugs"  had  been  worked  out  of  the  system.  In  general,  the  house  staff 
(residents  and  interns)  who  take  care  of  the  inpatients,  tended  to  be 
very  pleased  with  the  faster  availability  of  results  via  terminal 
inquiry,  thereby  avoiding  the  considerable  number  of  phone  calls  that 
were  previously  necessary.  Clinic  physicians  tended  to  have  mixed 
reactions  depending  on  the  extent  to  which  they  relied  on  terminal 
inquiry,  versus  review  of  hard-copy  results.  Some  clinic  physicians 
found  it  easier  to  use  the  terminal  to  look  up  patients'  results, 
whereas  other  physicians  preferred  to  utilize  hard-copy  reports. 

The  following  benefits  were  cited  by  physicians: 


Reduction  in  Telephone  Calls.  The  number  of  telephone 


calls  to  the  laboratory  had  dropped  considerably.  It 
was  now  necessary  to  call  only  when  either  the  computer 
was  down,  or  they  needed  information  for  patients  who  were 
seen  more  than  30  days  ago. 

Improved  Turnaround  Time.  Hard-copy  reports  did  get  back 
sooner  but  this  did  not  mean  that  the  outpatient  chart  was 
more  often  complete  (presumably  due  to  the  workload  in 
the  Medical  Records  Department) . 
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•  Improved  Quality  of  Care.  Quality  of  care  had  improved 
because  of  the  report  format,  including  flagging  of 
abnormal  values  and  inclusion  of  some  test  results  in 
the  daily  cumulative  summary  report  which  were  not 
included  in  the  flow  sheets.  Routine  values  were  now 
available  earlier  (via  terminal  inquiry)  than  before 
TRILAB . 

To  summarize,  both  the  interviews  and  the  questionnaire  survey 
(Ill-Section  D)  indicated  that  health  care  providers  were  generally 
pleased  with  the  TRILAB  system,  citing  as  advantages  reduced  telephone 
calls,  decreased  turnaround  times,  improvements  in  relationships  with 
laboratory  personnel,  and  improvement  in  quality  of  care  due  to  easier 
and  faster  access  to  test  results,  identification  of  abnormal  values, 
and  cumulative  report  formats.  A  further  indirect  measure  of  approval 
of  the  system  was  the  expressed  desire  of  staff  in  those  inpatient 
units  and  outpatient  clinics  that  did  not  have  terminals  (and  had  to 
share  a  terminal  in  another  location)  for  a  terminal  in  their  own 
location. 

As  indicated  in  Section  D,  the  major  problems  or  improvements 
identified  by  providers  were: 

•  addition  of  Nuclear  Medicine  results  to  the  system; 

•  improving  the  format  of  reports  to  reduce  paper  volume;  and 

•  reducing  computer  down-time  disruptions. 

4.  Benefits  Achieved  in  Laboratory 

The  following  discussion  of  benefits  of  the  TRILAB  system  to 
laboratory  staff  is  based  on  interviews  carried  out  with  laboratory 
administration  and  supervisors  during  the  implementation  monitoring 
and  post-implementation  period  study  data  collection  visits. 
Laboratory  staff  identified  the  following  benefits  from  the  system: 

•  Reduced  Telephone  Calls.  It  was  estimated  that  telephone 
calls  to  the  laboratory  had  been  reduced  by  about  50  percent, 
because  of  availability  of  results  and  test  status  to 
providers  via  terminal  inquiry.  The  study  of  telephone 

call  volume  (Section  III-2)  indicated  that  telephone 
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call  volume  decreased  by  43  percent.  This  resulted  in 
less  time  in  actually  answering  calls  and  a  reduction 
in  the  disruptions  in  the  laboratory  work  flow. 

•  Workload  Reporting.  Supervisors  estimated  that  in  the 
three  major  sections  (Chemistry,  Hematology,  and 
Microbiology),  approximately  8.5  hours  per  week  in  total 
were  saved  by  having  the  TRILAB  system  produce  the  monthly 
workload  reports;  these  reports  were  previously  prepared 
manually.  (This  compares  with  the  reduction  of  21  hours 
estimated  using  the  work  sampling  study  data.) 

•  Quality  Control  Reports.  Supervisors  similarly  estimated 
that  11  hours  per  week  were  saved  in  producing  the  quality 
control  reports,  which  were  produced  by  the  TRILAB 
computer  system.  (This  compares  with  31  hours  per  week 
saved  estimated  using  the  work  sampling  study  data.) 

•  Patient  Exception  Reports.  It  was  estimated  that 
approximately  10  hours  per  week  were  saved  in  review 
of  patient  exception  reports,  due  to  the  highlighting 
of  abnormal  results  by  the  computer  system. 

•  Reception  Desk.  As  a  result  of  removal  of  the  results 
files  from  the  reception  area,  and  the  fewer  number  of 
telephone  calls  to  the  laboratory,  it  was  possible  to 
reduce  the  number  of  staff  at  the  reception  area  from 
1.5-2  to  1-1.5;  that  is,  a  savings  of  0.5  FTE  was 
achieved  at  the  reception  area. 

•  Duplicate  Tests.  The  number  of  duplicate  or  repeated 

tests  have  been  reduced  with  the  TRILAB  system  for  two 
reasons:  (1)  providers  could  easily  check  on  the  status 

of  tests,  and  would  be  less  Inclined  to  repeat  an  order 

if  they  saw  a  test  was  pending  or  in  process;  (2)  abnormal 
results  and  unusual  "delta  checks"  (abrupt  changes  from 
the  previous  day's  results)  showed  up  on  the  screen  as  they 
were  entered,  so  that  extra  attention  was  given  to  such 
results,  which  may  have  resulted  in  fewer  result  report 
errors. 
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No  data  were  available  on  the  frequency  of  duplicate  or 
repeat  tests,  but  supervisors  estimated  that  there  might 
be  a  reduction  of  approximately  one  percent  in  total  tests 
because  of  the  improvement.  This  would  approximate  a 
reduction  of  about  450  tests  per  week  based  on  the  current 
workload.  This  compares  with  the  reduction  of  40  or  50 
duplicate  tests  per  week  avoided,  estimated  by  inpatient 
nursing  staff  (Section  1  above).  To  be  conservative,  it 
is  estimated  that  100  duplicate  tests  per  week  may  be 
avi  ^ed,  representing  about  0.2  percent  reduction  in  total 
tests. 

In  addition  to  the  above  (quantifiable)  estimates  of  benefits, 
laboratory  staff  cited  the  following  benefits: 

•  Reduction  in  Transcription  Errors.  Because  abnormal 
results  were  highlighted  on  the  CRT  screens,  and  received 
extra  scrutiny  by  technicians  and  reviewers,  there  was 
potential  for  reduction  of  transcription  errors.  In 
addition,  the  Microbiology  section  had  found  that  the 
computer-generated  "infection  control  report,"  which 
lists  Microbiology  results  by  patient,  had  enabled  the 
section  to  identify  result  report  format  errors.  These 
errors  were  mostly  clerical — that  is,  where  the  wording 
of  results  could  have  been  improved;  only  one  "blatant" 
error  had  been  detected  during  the  period  of  system 
operation.  Thus  this  review  capability,  which  was  not 
available  in  the  manual  system,  was  used  mainly  to 
educate  technicians  to  improve  their  reporting  of 
Microbiology  results. 

•  Normal  Ranges  Data.  The  system  provided  the  capability 
to  provide  normal  ranges  data  with  each  test  (which  is  a 
CAP  accreditation  requirement).  This  was  not  provided 
previously,  at  least  for  the  majority  of  tests  for  which 
the  users  were  expected  to  know  what  the  normal  ranges  were. 
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•  Search  Capability.  The  availability  of  the  computerized  data 
base  of  test  results  and  associated  demographic  data,  provides 
the  potential  capability  of  performing  a  variety  of  analyses 
with  regard  to  utilization,  epidemiologic  analysis,  etc.  This 
capability  has  not  been  utilized  as  yet. 

•  Management  Reporting.  At  the  time  of  the  post-implementation 
study,  the  workload  reporting  system  was  being  enhanced  to 
provide  a  more  detailed  analysis  of  workload  by  section, 
shift  and  day  of  week,  and  analysis  of  workload  per  assigned 
(FTE)  staffing.  This  would  enable  laboratory  management 

to  improve  the  allocation  of  staffing  resources  in  response 
to  workload,  and  thereby  improve  the  overall  efficiency 
and  effectiveness  of  laboratory  services. 

B.  RECOMMENDATIONS  TO  ENHANCE  BENEFITS 

The  following  section  presents  suggestions  on  potential 
improvements  to  the  TRILAB  system,  based  on  observations  and 
interviews  with  data  processing,  laboratory,  and  provider  staff  at 
NRMC  Oakland.  These  suggestions  may  be  useful  for  other  institutions, 
as  well,  for  which  implementation  of  the  system  is  being  considered. 

1.  Memory  Capacity 

One  of  the  major  difficulties  with  the  TRILAB  system  at  NRMC 
Oakland  was  that  the  on-line  memory  capacity  of  the  computer  system 
appeared  to  be  significantly  undersized.  This  evidenced  itself  in  a 
number  of  ways: 

•  It  was  possible  to  maintain  only  30  days  of  patient  results 
data  on  the  system  for  on-line  inquiry,  compared  with  the 
original  specification  for  6  months  of  patient  results.  As 
a  result,  physicians  reported  difficulty  in  retrieving 
results  for  long-term  patients  who  were  seen  at  intervals 
of  more  than  30  days.  Also,  laboratory  staff  spent  addi¬ 
tional  time  in  loading  in  patient  identification  data  for 
prior  patients  who  had  been  removed  fiom  the  active  file. 

•  System  response  was  degraded  at  certain  hea\ry-use  times  of 
the  day,  usually  between  0900  and  1100,  and  between  1300 
and  1500  hours.  The  main  problems  were  felt  in  the 
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laboratory,  where  test  acqulsitioning  and  result  inputting 
were  slowed  down.  Providers  also  found  that  response  time 
to  inquiries  was  degraded. 

•  The  times  of  the  day  during  which  the  line  printer  may  be 
used  to  print  major  reports  was  severely  restricted,  in 
order  that  the  other  system  functions  not  be  affected. 
(Improving  the  printing  capacity  may  require  a  combination 
of  a  higher  speed  printer,  increased  memory,  and  changes 
in  software.) 

2.  Report  Formats 

The  number  and  format  of  reports  has  gone  through  some  evolution 
since  original  implementation  of  the  system.  As  is  often  the  case 
with  implementation  of  computer  systems,  once  users  become  familiar 
with  the  capability  of  the  system  they  tend  to  ask  for  additional  and 
more  detailed  data  and  reports.  After  a  period  of  time,  however,  they 
find  that  some  of  the  additional  data  or  reports  are  redundant  or  not 
needed.  In  particular,  many  users  commented  on  the  fact  that  some  of 
the  reports  should  be  more  concise  in  order  to  eliminate  the  volume  of 
paper,  and  that  some  data  could  be  usefully  omitted.  It  would  be 
useful  to  review  with  users  their  specific  needs,  in  order  to 
determine  whether  any  of  the  reports  could  be  improved  in  format,  or 
reduced  in  content. 

3.  Need  for  More  Terminals 


Again  it  is  not  a  surprising  result  that,  after  usefulness  of  the 
system  has  been  demonstrated,  particularly  results  reporting  and 
inquiry  capabilities,  those  user  locations  which  did  not  have 
terminals  should  desire  one.  In  addition,  it  may  be  useful  to 
consider  terminals  for  remote  locations  served  by  the  laboratory,  in 
order  to  improve  their  report  results  turnaround  times. 

4.  Operational  Problems 

Although  the  TRILAB  system  functioned  in  accordance  with 
expectations,  there  were  a  number  of  problems  which  persisted,  which 
if  eliminated  would  reduce  some  frustrations  associated  with  system 
operations.  These  include,  for  example,  inappropriate  beeping  of 
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terminals,  repeat  results  reporting,  and  systems'  maintenance  being 
carried  out  during  the  daytime.  Continued  attention  to  solving  such 
problems  will  improve  its  acceptance  and  usefulness  and  reduce  any 
frustrations  experienced  with  the  system. 

C.  CONCLUSIONS 

The  analysis  of  the  data  collected  for  this  evaluation  suggests 
that  the  TRILAB  system  has  generally  met  the  overall  objectives  and 
evaluation  criteria  originally  developed  for  the  system.  Hospital 
staff  are  generally  pleased  with  the  system,  with  providers  expressing 
greater  satisfaction  and  receiving  greater  benefits  than  laboratory 
personnel . 

The  major  benefits  of  the  system  were: 

•  Improved  turnaround  time  for  test  results,  most  markedly 
in  availability  of  results  for  routine  Chemistry  and 
Hematology  tests. 

•  A  reduction  in  time  spent  by  inpatient  and  clinic  staff 
in  telephoning  for  test  results  and  test  status.  This 
reduction  in  staff  time  was  estimated  to  total  76  staff 
hours  per  day,  equivalent  to  9.5  FTEs,  made  available 
for  other  activities. 

•  Laboratory  results  were  more  easily  available  to  providers 
(via  terminal  inquiry)  and  in  more  useful  format  (provision 
of  cumulative  reports  and  highlighting  abnormal  results) , 
which  may  improve  quality  of  patient  care. 

Indirect  indicators  of  approval  of  the  system  by  providers  were 
the  requests  for  terminals  for  those  locations  which  did  not  have 
them,  and  that  Nuclear  Medicine  (which  is  not  now  on  the  system)  be 
added  as  soon  as  possible. 

The  benefits  to  laboratory  personnel  included  a  reduction  in 
staff  time  devoted  to  information  handling  activities,  a  reduction  in 
telephone  calls  to  the  laboratory,  and  increased  capability  to  improve 
efficiency  and  effectiveness  of  service  through  the  vorkload/- 
management  and  quality  control  reporting  capabilities. 
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D.  COMPARISON  OF  RESULTS  WITH  PROJECT  OBJECTIVES 


On  July  15,  1977,  the  TRIMIS  Medical  Review  Group  (MRG)  developed 
seven  project  objectives  for  the  TRI-Service  Laboratory  System^.  In 
this  section,  the  results  obtained  in  the  evaluation  at  NRMC  Oakland 
are  related  to  these  original  objectives. 

1.  To  Make  Information  Available  to  Physicians  with  Increased 
Efficiency  and  Accuracy 

Providers  reported  that  turnaround  time  for  test  results  had 
decreased  with  the  TRILAB  system;  this  was  confirmed  in  the  analysis 
of  turnaround  times.  The  turnaround  study  showed  that  for  routine 
tests,  results  were  available  to  provider  locations  in  less  time  for 
Chemistry  and  Hematology  tests,  and  in  about  the  same  time  period  or 
sooner  for  Microbiology.  For  STAT/urgent  tests,  process  times  were 
either  unchanged  (Chemistry)  or  reduced. 

Tests  repeated  due  to  delays  or  to  lost  results,  and  telephone 
calls  to  the  laboratory  were  also  reported  to  occur  with  less 
frequency  under  the  operation  of  the  TRILAB  system  than  with  the 
previous  system.  These  indicators  suggest  that  information  was  being 
made  available  to  physicians  with  increased  efficiency.  Data  were  not 
available  to  make  a  comparison  of  accuracy  of  results;  laboratory 
personnel  believed,  however,  that  accuracy  of  results  may  have  been 
improved  because  of  highlighting  of  unusual  results  values  by  the 
system,  facilitating  review  of  such  results.  Most  providers  felt, 
however,  that  repeating  of  tests  due  to  inaccurate  results  occurred 
with  similar  frequency  with  the  TRILAB  system  as  before. 

Providers,  especially  those  on  inpatient  services,  were  very 
satisfied  with  the  ability  of  retrieving  patient  laboratory  results 
via  terminal  inquiry,  and  with  the  cumulative  results  reports.  In 
some  cases  the  cumulative  reports  were  used  to  supplement  the  manual 
"flow  sheets,"  and  in  others  had  replaced  them.  Nursing  personnel 
estimated  that,  on  average,  four  staff  hours  per  inpatient  unit  per 
day  had  been  saved,  through  reductions  in  time  on  the  telephone, 
filing  time,  and  chart  review.  In  addition,  an  average  4.6  hours  of 
professional  and  clinical  staff  time  per  clinic  per  day  were  estimated 
to  be  saved. 


A  Arthur  D.  Little,  Inc. 


89 


(139  641  EVALUATION  OF  THE  TRI-SERVICE  LABORRTORV  SVSTEH  VOLUHE 
2  NAVAL  REGIONAL  N.  .  (U>  LITTLE  (ARTHUR  D>  INC  CAMBRIDGE 
NA  20  JUN  83  ADL-0209-2-LAB-1-V-FINRL-VOL-2 
ASSIFIED  NDA903-81-C-0289  F/G  6/5 


2/2 

NL 


5.  To  Gather  as  a  Result  of  Normal  Procedures,  Workload  and 
Managerial  Data,  and  to  Present  This  as  Required  in  Order 
to  Assist  in  Decision-Making  in  the  Laboratory 

The  system  gathered  and  presented  workload  data  (thereby  reducing 
the  time  devoted  to  this  activity) .  The  workload  report  was  being 
modified  to  enhance  its  capability  to  provide  workload  and  managerial 
data,  which  should  lead  to  improved  capability  of  allocating 
laboratory  resources  in  response  to  required  workload. 

6.  To  Reduce  the  Clerical  Work  Required  of  Qualified  Technicians 
in  the  Laboratory 

The  work  sampling  results  indicate  that,  overall,  time  devoted  to 
information  handling  activities  was  about  6  percent  less  in  the 
post-implementation  period  compared  with  the  baseline  period;  this 
difference  was  about  2.6  percent  of  total  laboratory  technician  staff 
time.  Laboratory  personnel  indicated  that  identification  of 
incomplete  and  pending  tests  and  results  was  improved  by  the  TRILAB 
system  and  that  time  spent  on  manual  record-keeping  occurred  less 
frequently  with  the  TRILAB  system.  They  reported  that  "efficiency  of 
laboratory  operations"  and  "ease  of  information  storage  and  retrieval" 
were  either  "very  important"  or  "somewhat  important"  improvements  due 
to  TRILAB. 

The  major  change  in  non-personnel  operating  costs  in  the 
laboratory  was  likely  due  to  a  reduction  in  duplicated  tests,  and  the 
associated  reagent  costs.  Providers  and  laboratory  staff  estimated 
that  the  reduction  might  have  been  approximately  100  tests  per  week. 

7.  To  Improve  Result  Accuracy  by  Eliminating  Transcription, 
Calculation,  and  Specimen  Identification  Error 

No  data  were  available  to  measure  this  effect.  Both  providers 
and  laboratory  staff,  however,  believed  that  such  errors  have  been 
reduced,  resulting  in  fewer  duplicate  tests  requires. 

It  is  concluded  that  the  original  project  goals  have  been  by  and 
large  met,  albeit  modestly  for  the  objectives  of  enabling  the 
laboratory  to  handle  increased  demands  for  testing  and  for  reducing 
the  clerical  work  required  to  laboratory  technicians. 
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